Category Archives: Programming

Subversion diff from url to local directory working copy

For some reason there is some bad information out there on comparing a local working copy in a branch to that of a subversion url.  In this case, I really wanted to know how far off this branch code that wasn’t commited is from the trunk.  To do that, you just need to use the ‘–old’ and ‘–new’ flags.

$ cd ./branches/one
$ svn diff --old=svn://server/Project/trunk --new=. > working.diff

Then all that needs to be done to update a branch of trunk to that is:

$ patch -p0 -i workingdiff.diff

My IOIO for Android is here

The IOIO for Android that I recently purchased has shown up!!!  I haven’t had time to play with it or even know much about how to start playing with it, but I will be figuring it out quickly.  If you’re not sure what it is, a quick description from their website is:

The IOIO (pronounced “yo-yo”) is a board specially designed to work with your Android 1.5 and later device. The board provides robust connectivity to an Android device via a USB or Bluetooth connection and is fully controllable from within an Android application using a simple and intuitive Java API - no embedded programming or external programmer will ever be needed!

So setup as it’s default use, it’s not using the new Google Open Accessory (ADK) protocol, which is what I really want to develop with once I get far enough along in Android programming and being able to do some prototypes on the board.  I figure once I get far enough along, that I’ll be turning on blinking LED’s, and from there the worlds the limit, even though I do know what I want the end result to be with it.

The bad news at this point is that the ADK firmware on it is still very beta.  Even to update the device, I need a PIC programmer capable of programming PIC24FJ128DA206.  So that means until it comes out of beta, I can’t use it for the ADK protocol or I ask some EE friends if they have one laying around.  Even if I take the friend route to update it to the beta, it still has the following problematic caveat:

Due to problems with the accessory library implementation (i.e. the Android OS code supporting the protocol) – the IOIO connection will not be properly closed when pausing (and exiting) your IOIO app. When the app is resumed (restarted) it will hang. The current workaround is to physically disconnect and reconnect the IOIO (or power-cycle it).

This is because of a problem in the Android OpenAccessory library preventing the app from gracefully closing the ADK connection. To work around this problem, detach and re-attach the IOIO when that happens (or power-cycle it). Another option is to force-close the app.

Not something that I would call usable in a real life situation.  Hopefully it does get resolved though.  At this point though, since my original plan is for a single use at this point, if all else fails, I’ll just use their implementation of the communication protocol for the initial project.  Either library I use in the end, both will be good learning experiences on making and developing for new accessories to work with Android.

Using regex on Syslog-ng to save to specific file locations

I had the problem of taking an external syslog feed (through UDP or TCP) that came from multiple devices and then separating them into folders that identified the feeds by customers / device type / host. The one thing I had on my side was that the hostname of the devices were broken up into three parts to help me get this information.

In the old Syslog-ng (1.6.x), there was not a way to regex out this information and then use it in the destination section. Each time we had a new combination, it required writing additional lines with additional regexs. Started to really feel it on performance. With the new branch of syslog-ng (2.0.x), this feature is now available.

You can use up to 256 different $NNN ($1 … $256) macros, but you can only use one regexp expression. This will be done using a filter.

Here’s a full example of what I had done then. What I wanted to do is put the line in a specific file based on part of the hostname. For example:

Hostname: subdomain.domain.tld
Save the log files for that device at:
/logs/$TLD/$DOMAIN/$SUBDOMAIN/$R_YEAR-$R_MONTH-$R_DAY.log

WHERE $TLD is the tld of the hostname, $DOMAIN is the domain from the hostname, and $SUBDOMAIN is the subdomain from the hostname.

I would like to be able to regex this information out of the hostname to use in the destination.

To accomplish this, I was able to use this configuration:

filter f_filter { host(“^([0-9a-zA-Z\-]+)\.([0-9a-zA-Z\-]+)\.([0-9a-zA-Z\-]+)$”); };
destination f_logs { file(“/logs/$3/$2/$1/$YEAR-$MONTH-$DAY.log”); };
log {
     source(external);
     filter(f_filter);
     destination(f_logs);
     flags(final);
};

Crypt::SSLeay on Solaris 10

I had to do some development in perl that requied scrubbing a HTTPS page.  I ran into an issue when I moved the script from the laptop to the production solaris server that it was not able to process the HTTPS page.  Digging down a little deeper, I realized I never installed one of the required SSL modules for perl to allow libwww modules to connect to https pages.

You could actually use either Crypt::SSLeay or IO::Socket::SSL (which also required Net::SSLeay).  I decided to install them all to be on the safe side.  Net::SSLeay and IO::Socket::SSL had no problems, but when I tried to make Crypt::SSLeay, I ran into an issue.

When I tried to make the Makefile, I did the following:

root@appsrv /opt/downloads/cpan/Crypt-SSLeay-0.55 # perl Makefile.PL
Skipping testcover target, ExtUtils::MakeMaker::Coverage not found
=======================================================
Only one OpenSSL installation found at /usr/local/ssl
Consider running ‘perl Makefile.PL –default’ the next
time Crypt::SSLeay is upgraded to select this directory
automatically thereby avoiding the following prompt.
=======================================================
Which SSL install path do you want to use? [/usr/local/ssl]

BUILD INFORMATION
================================================
ssl library: OpenSSL 0.9.8 in /usr/local/ssl
ssl header:  openssl/ssl.h
libraries:   -L/usr/local/ssl/lib -lssl -lcrypto -lgcc
include dir: -I/usr/local/ssl/include/openssl
================================================
WARNING: LICENSE is not a known parameter.
‘LICENSE’ is not a known MakeMaker parameter name.
Writing Makefile for Crypt::SSLeay
The test suite can attempt to connect to public servers
to ensure that the code is working properly. If you are
behind a strict firewall or have no network connectivity,
these tests may fail (through no fault of the code).

Do you want to run the live tests (y/N) ? [N]
root@appsrv /opt/downloads/cpan/Crypt-SSLeay-0.55 # make
gcc -B/usr/ccs/bin/ -c  -I/usr/local/ssl/include/openssl -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O   -DVERSION=\"0.55\" -DXS_VERSION=\"0.55\" -fPIC "-I/usr/local/lib/perl5/5.8.7/sun4-solaris/CORE"   SSLeay.c
In file included from SSLeay.xs:25:
crypt_ssleay_version.h:1:25: openssl/ssl.h: No such file or directory
crypt_ssleay_version.h:2:28: openssl/crypto.h: No such file or directory
crypt_ssleay_version.h:3:25: openssl/err.h: No such file or directory
crypt_ssleay_version.h:4:26: openssl/rand.h: No such file or directory
crypt_ssleay_version.h:5:28: openssl/pkcs12.h: No such file or directory
SSLeay.xs:43: error: syntax error before ‘*’ token
SSLeay.xs: In function `InfoCallback’:
SSLeay.xs:48: error: `where’ undeclared (first use in this function)
SSLeay.xs:48: error: (Each undeclared identifier is reported only once
SSLeay.xs:48: error: for each function it appears in.)
SSLeay.xs:48: error: `SSL_ST_MASK’ undeclared (first use in this function)
SSLeay.xs:50: error: `SSL_ST_CONNECT’ undeclared (first use in this function)
SSLeay.xs:52: error: `SSL_ST_ACCEPT’ undeclared (first use in this function)
SSLeay.xs:57: error: `SSL_CB_LOOP’ undeclared (first use in this function)
SSLeay.xs:58: error: `s’ undeclared (first use in this function)
SSLeay.xs:60: error: `SSL_CB_ALERT’ undeclared (first use in this function)
SSLeay.xs:61: error: `SSL_CB_READ’ undeclared (first use in this function)
SSLeay.xs:63: error: `ret’ undeclared (first use in this function)
SSLeay.xs:66: error: `SSL_CB_EXIT’ undeclared (first use in this function)
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_new’:
SSLeay.c:118: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:118: error: `RETVAL’ undeclared (first use in this function)
SSLeay.xs:102: error: `ctx’ undeclared (first use in this function)
SSLeay.xs:135: error: `SSL_OP_ALL’ undeclared (first use in this function)
SSLeay.xs:137: error: `SSL_VERIFY_NONE’ undeclared (first use in this function)
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_free’:
SSLeay.c:172: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:172: error: `ctx’ undeclared (first use in this function)
SSLeay.c:176: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_set_cipher_list’:
SSLeay.c:193: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:193: error: `ctx’ undeclared (first use in this function)
SSLeay.c:200: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_use_certificate_file’:
SSLeay.c:218: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:218: error: `ctx’ undeclared (first use in this function)
SSLeay.c:226: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_use_PrivateKey_file’:
SSLeay.c:244: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:244: error: `ctx’ undeclared (first use in this function)
SSLeay.c:252: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_use_pkcs12_file’:
SSLeay.c:270: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:270: error: `ctx’ undeclared (first use in this function)
SSLeay.xs:170: error: `EVP_PKEY’ undeclared (first use in this function)
SSLeay.xs:170: error: `pkey’ undeclared (first use in this function)
SSLeay.xs:171: error: `X509′ undeclared (first use in this function)
SSLeay.xs:171: error: `cert’ undeclared (first use in this function)
SSLeay.xs:172: error: `ca’ undeclared (first use in this function)
SSLeay.xs:173: error: `PKCS12′ undeclared (first use in this function)
SSLeay.xs:173: error: `p12′ undeclared (first use in this function)
SSLeay.c:285: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_check_private_key’:
SSLeay.c:321: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:321: error: `ctx’ undeclared (first use in this function)
SSLeay.c:327: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_set_verify’:
SSLeay.c:345: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:345: error: `ctx’ undeclared (first use in this function)
SSLeay.c:354: error: syntax error before ‘)’ token
SSLeay.xs:212: error: `SSL_VERIFY_NONE’ undeclared (first use in this function)
SSLeay.xs:217: error: `SSL_VERIFY_PEER’ undeclared (first use in this function)
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_new’:
SSLeay.c:386: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:386: error: `ctx’ undeclared (first use in this function)
SSLeay.xs:231: error: `SSL’ undeclared (first use in this function)
SSLeay.xs:231: error: `ssl’ undeclared (first use in this function)
SSLeay.c:391: error: `RETVAL’ undeclared (first use in this function)
SSLeay.c:395: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_free’:
SSLeay.c:440: error: `SSL’ undeclared (first use in this function)
SSLeay.c:440: error: `ssl’ undeclared (first use in this function)
SSLeay.c:444: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_set_fd’:
SSLeay.c:461: error: `SSL’ undeclared (first use in this function)
SSLeay.c:461: error: `ssl’ undeclared (first use in this function)
SSLeay.c:468: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_connect’:
SSLeay.c:486: error: `SSL’ undeclared (first use in this function)
SSLeay.c:486: error: `ssl’ undeclared (first use in this function)
SSLeay.c:492: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_accept’:
SSLeay.c:510: error: `SSL’ undeclared (first use in this function)
SSLeay.c:510: error: `ssl’ undeclared (first use in this function)
SSLeay.c:516: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_write’:
SSLeay.c:534: error: `SSL’ undeclared (first use in this function)
SSLeay.c:534: error: `ssl’ undeclared (first use in this function)
SSLeay.c:546: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_read’:
SSLeay.c:590: error: `SSL’ undeclared (first use in this function)
SSLeay.c:590: error: `ssl’ undeclared (first use in this function)
SSLeay.c:603: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_get_peer_certificate’:
SSLeay.c:655: error: `SSL’ undeclared (first use in this function)
SSLeay.c:655: error: `ssl’ undeclared (first use in this function)
SSLeay.c:656: error: `X509′ undeclared (first use in this function)
SSLeay.c:656: error: `RETVAL’ undeclared (first use in this function)
SSLeay.c:660: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_get_verify_result’:
SSLeay.c:679: error: `SSL’ undeclared (first use in this function)
SSLeay.c:679: error: `ssl’ undeclared (first use in this function)
SSLeay.c:684: error: syntax error before ‘)’ token
SSLeay.xs:373: error: `X509_V_OK’ undeclared (first use in this function)
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_get_shared_ciphers’:
SSLeay.c:704: error: `SSL’ undeclared (first use in this function)
SSLeay.c:704: error: `ssl’ undeclared (first use in this function)
SSLeay.c:713: error: syntax error before ‘)’ token
SSLeay.xs:383: warning: assignment makes pointer from integer without a cast
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_get_cipher’:
SSLeay.c:732: error: `SSL’ undeclared (first use in this function)
SSLeay.c:732: error: `ssl’ undeclared (first use in this function)
SSLeay.c:738: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__X509_free’:
SSLeay.c:757: error: `X509′ undeclared (first use in this function)
SSLeay.c:757: error: `cert’ undeclared (first use in this function)
SSLeay.c:761: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__X509_subject_name’:
SSLeay.c:778: error: `X509′ undeclared (first use in this function)
SSLeay.c:778: error: `cert’ undeclared (first use in this function)
SSLeay.c:786: error: syntax error before ‘)’ token
SSLeay.xs:407: warning: assignment makes pointer from integer without a cast
SSLeay.c: In function `XS_Crypt__SSLeay__X509_issuer_name’:
SSLeay.c:808: error: `X509′ undeclared (first use in this function)
SSLeay.c:808: error: `cert’ undeclared (first use in this function)
SSLeay.c:816: error: syntax error before ‘)’ token
SSLeay.xs:419: warning: assignment makes pointer from integer without a cast
SSLeay.c: In function `XS_Crypt__SSLeay__X509_get_notBeforeString’:
SSLeay.c:838: error: `X509′ undeclared (first use in this function)
SSLeay.c:838: error: `cert’ undeclared (first use in this function)
SSLeay.c:844: error: syntax error before ‘)’ token
SSLeay.xs:429: error: invalid type argument of `->’
SSLeay.c: In function `XS_Crypt__SSLeay__X509_get_notAfterString’:
SSLeay.c:863: error: `X509′ undeclared (first use in this function)
SSLeay.c:863: error: `cert’ undeclared (first use in this function)
SSLeay.c:869: error: syntax error before ‘)’ token
SSLeay.xs:437: error: invalid type argument of `->’
make: *** [SSLeay.o] Error 1
root@appsrv /opt/downloads/cpan/Crypt-SSLeay-0.55 #

I checked and made sure that the LD_LIBRARY_PATH had the openssl libraries in it, which it did.  I then went into the generated Makefile and changed line 147 from

INC = -I/usr/local/ssl/include/openssl

to

 INC = -I/usr/local/ssl/include

Which fixed the problem.  I was then able to make and install the module. 

MySQL DST Patch / Update

To update the MySQL server,  you really do not have to run a patch.  That is a common question I was getting from people was where to get the patch.  You actually have to patch the OS and then from there run a command on MySQL.  If you’re OS doesn’t use zoneinfo, there is even a fix for this without having to reinstall a newer version of MySQL.

Taken from http://dev.mysql.com/doc/refman/5.0/en/time-zone-support.html:

If your system has its own zoneinfo database (the set of files describing time zones), you should use the mysql_tzinfo_to_sql program for filling the time zone tables. Examples of such systems are Linux, FreeBSD, Sun Solaris, and Mac OS X. One likely location for these files is the /usr/share/zoneinfo directory. If your system does not have a zoneinfo database, you can use the downloadable package described later in this section.

The mysql_tzinfo_to_sql program is used to load the time zone tables. On the command line, pass the zoneinfo directory pathname to mysql_tzinfo_to_sql and send the output into the mysql program. For example:

shell&gt; <strong class="userinput">mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql</strong>   

mysql_tzinfo_to_sql reads your system’s time zone files and generates SQL statements from them. mysql processes those statements to load the time zone tables.

mysql_tzinfo_to_sql also can be used to load a single time zone file or to generate leap second information:

  • To load a single time zone file tz_file that corresponds to a time zone name tz_name, invoke mysql_tzinfo_to_sql like this:

    shell&gt; <strong class="userinput">mysql_tzinfo_to_sql <em class="replaceable">tz_file</em> <em class="replaceable">tz_name</em> | mysql -u root mysql</strong> 

    With this approach, you must must execute a separate command to load the time zone file for each named zone that the server needs to know about.

  • If your time zone needs to account for leap seconds, initialize the leap second information like this, where tz_file is the name of your time zone file:

    shell&gt; <strong class="userinput">mysql_tzinfo_to_sql --leap <em class="replaceable">tz_file</em> | mysql -u root mysql</strong> 

If your system is one that has no zoneinfo database (for example, Windows or HP-UX), you can use the package of pre-built time zone tables that is available for download at the MySQL Developer Zone:

<a target="_top" href="http://dev.mysql.com/downloads/timezones.html">http://dev.mysql.com/downloads/timezones.html</a>   

This time zone package contains .frm, .MYD, and .MYI files for the MyISAM time zone tables. These tables should be part of the mysql database, so you should place the files in the mysql subdirectory of your MySQL server’s data directory. The server should be stopped while you do this and restarted afterward.

Warning: Do not use the downloadable package if your system has a zoneinfo database. Use the mysql_tzinfo_to_sql utility instead. Otherwise, you may cause a difference in datetime handling between MySQL and other applications on your system.

Certified MySQL Developer

Just got the results in from MySQL … I’ve passed both the Certified MySQL Developer Part I and Part II exams.  I took one exam in March and the other in April, but both were beta exams.  That meant that I was not able to get results at the end of the exam.  I’ll put the certified logo up on this post once MySQL releases a certified logo.

MySQL5_0Developer.png 

Approx Date for MySQL Beta Certification Exam 5.0 Results

I had taken the MySQL Beta Developer 5.0 Certificatino Exam a couple weeks ago.  Since it was a beta, I knew I would get my results for a good 8 – 12 weeks after the beta exams were supposed to end, which at that time was April 15th.  I stopped over at the MySQL website today, to get the information on the JDBC for MySQL.  That’s when I noticed that the beta exams were extended until May 15th.  I guess their website was not finished being updated, because a lot of places on the page still listed it as ending April 15th.

A bit frustrated with the new date, because I did not want to wait an additional 8 – 12 weeks after May 15th, I sent them an email asking when they intended for the results to be sent.  I was hoping that it was still inteded for 8 – 12 weeks after April 15th.  I quickly received an email from Carsten informing me that they will be grading the exams right after May 15th and that he strongly suspects that I will see the results from my certification exam before the end of May.  Woohoo.

Since the certification has been extended another month, I think I might consider taking the first certification exam for the MySQL 5.0 Administrator certification.  Haven’t decided on that though since I also want to get some Java certifications soon. 

PHP, SSL, File Output and Internet Explorer Error

While working on a project, everything was working on our beta site just fine.  Once we moved the project to the production site, we ran into an issue on a page that exported a csv file of information inline.  The only difference was that the production site was running SSL.

The basic situation is as follows.  A script written on PHP, running on Apache, through a SSL connection is dynamically generating a file (in our case a csv) and then sending it to the client.  This works correctly through Firefox, but returns an error in Internet Explorer.  The error returned is:

Internet Explorer cannot download <url of link>

Internet Explorer was not able to open this Internet site.  The requested site is either unavailable or cannot be found.  Please try again later.

Here’s a hint, no matter how many times you try again later, it will not work.  I started with this code:

header("Content-type: application/csv\n");
header("Content-Disposition: attachment; filename=\"$filename\"; size=$size");

After much research, I found the way to fix this is through the Pragma header, making sure it’s anything BUT off-cache.  Once I add a new header, it worked fine through both Firefox and Internet Explorer:

header("Content-type: application/csv\n");
header("Content-Disposition: attachment; filename=\"$filename\"; size=$size");
header("Pragma: turn-off-cache", true);

I just changed it to turn-off-cache, but anything typed in instead of off-cache would work fine.

Komodo on Gentoo AMD64

I had ran into a few issues when I tried running Komodo on my desktop machine that is running an AMD64.  The install went just fine, but when I tried to run Komodo it would fail while looking for libraries.  In my particular case it was the libgtk-x11-2.0.so.0.  When I did `ldd komodo-bin`, I realized that was not the only library that was missing.  It ended up I needed to install more lib32 compat libraries.  The fix to the issue was to emerge ‘app-emulation/emul-linux-x86-gtklibs’ which fixed all my problems and allowed me to run Komodo.

Here is a list of the emul libraries that I have installed on my system in case other people are having problems:

app-emulation/emul-linux-x86-baselibs
app-emulation/emul-linux-x86-compat
app-emulation/emul-linux-x86-glibc
app-emulation/emul-linux-x86-gtklibs
app-emulation/emul-linux-x86-java
app-emulation/emul-linux-x86-soundlibs
app-emulation/emul-linux-x86-xlibs

I’m not saying that you will need all of these.   More than likely you will not need them all.  This is just the list of the ones that I had installed when Komodo is working. 

ZCE 1000

Looks like Zend is celebrating their 1000 person to receive a ZCE (Zend Certified Engineer) certification.  They sent me an email about it a couple days ago, but I’ve been so busy that I really have not had a chance to look at it until now.  Apparently, they have created a new ZCE logo for the first 1000 ZCE’s to use.  I just happen to be one of the first 1000.  In case you’re not sure what the Zend Certified Engineer means, it basically is a certification for programming with PHP.

ZCE1000.GIF