James Whitt’s Blog

Well, I’ve managed to upgrade wordpress and gallery again. It’s been a while, so the versions were pretty far behind. One nice benefit of this upgrade is that the photo gallery is now working again. Enjoy.

I had the problem of taking an external syslog feed (through UDP or TCP) that came from multiple devices and then separating them into folders that identified the feeds by customers / device type / host. The one thing I had on my side was that the hostname of the devices were broken up into three parts to help me get this information.

In the old Syslog-ng (1.6.x), there was not a way to regex out this information and then use it in the destination section. Each time we had a new combination, it required writing additional lines with additional regexs. Started to really feel it on performance. With the new branch of syslog-ng (2.0.x), this feature is now available.

You can use up to 256 different $NNN ($1 … $256) macros, but you can only use one regexp expression. This will be done using a filter.

Here’s a full example of what I had done then. What I wanted to do is put the line in a specific file based on part of the hostname. For example:

Hostname: subdomain.domain.tld
Save the log files for that device at:
/logs/$TLD/$DOMAIN/$SUBDOMAIN/$R_YEAR-$R_MONTH-$R_DAY.log

WHERE $TLD is the tld of the hostname, $DOMAIN is the domain from the hostname, and $SUBDOMAIN is the subdomain from the hostname.

I would like to be able to regex this information out of the hostname to use in the destination.

To accomplish this, I was able to use this configuration:

filter f_filter { host(”^([0-9a-zA-Z\-]+)\.([0-9a-zA-Z\-]+)\.([0-9a-zA-Z\-]+)$”); };
destination f_logs { file(”/logs/$3/$2/$1/$YEAR-$MONTH-$DAY.log”); };
log {
     source(external);
     filter(f_filter);
     destination(f_logs);
     flags(final);
};

I had a disk fail while rebooting a solaris 8 machine. I had a RAID 1 setup with two disks using Disk Suite. The first drive in the boot order was the drive that failed. Since it failed during the reboot, when Solaris started back up, it started in single user mode. It gave the error:
Insufficient metadvice database replicas located

Here is some of the information that I found out:

root@server / # metastat
d0: Mirror
Submirror 0: d10
State: Needs maintenance
Submirror 1: d20
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 33002848 blocks

root@server / # metastat -p
d0 -m d10 d20 1
d10 1 1 c0t0d0s0
d20 1 1 c0t1d0s0
d1 -m d11 d21 1
d11 1 1 c0t0d0s1
d21 1 1 c0t1d0s1
root@server / # metadb -i
flags first blk block count
M p unknown unknown /dev/dsk/c0t0d0s3
M p unknown unknown /dev/dsk/c0t0d0s3
M p unknown unknown /dev/dsk/c0t0d0s4
M p unknown unknown /dev/dsk/c0t0d0s4
a m p lu 16 1034 /dev/dsk/c0t1d0s3
a p l 1050 1034 /dev/dsk/c0t1d0s3
a p l 16 1034 /dev/dsk/c0t1d0s4
a p l 1050 1034 /dev/dsk/c0t1d0s4


To fix this, do the following:

1. When you are booted into single user mode, delete any metadevice state database replicas that were on the ‘bad’ disk. This disk does not even have to been in to do this, but it needs to be done. So I had to do the following for my system:
# metadb -d /dev/dsk/c0t0d0s3
# metadb -d /dev/dsk/c0t0d0s4

2. Make sure that the metadevices have been deleted.
# metadb -i

3. Pull the ‘bad’ disk out and leave the bay empty. Reboot the machine.

4. At this point, you should be in normal boot sequence and get you into a running system, with only one disk in the RAID.

5. Put the replacement disk in the bay.

6. Partition the new disk. Easiest way to do this is to copy the partition table from the working disk.
# dd if=/dev/rdsk/c0t1d0s2 of=/dev/rdsk/c0t0d0s2 count=16

7. Recreate the metadevice state database replicas on the new disk. I had to do the following:
# metadb -a -f -c /dev/dsk/c0t0d0s3
# metadb -a -f -c /dev/dsk/c0t0d0s4

8. Setup the disks so that they are back in the submirrors:
# metareplace –e d0 c0t0d0s0
# metareplace –e d1 c0t0d0s1

At that point, you should have everything up and running. You can use ‘metastat’ to watch the disks sync up.

I’ve been doing installs of Solaris 10 and then needing to set them up with SNMP. The first thing I noticed is that the system is running snmpd and not snmpdx. There is still an init.d file for snmpdx, init.snmpdx, and configuration files, so at first this confused me. After looking around the web, I found that Solaris 10 is moving to using snmpd (Net-SNMP) instead of the solstice version.

Finding that information out, I was able to determine that the configuration files for snmpd are located in /etc/sma/snmp/. To setup snmpd information, just modify the snmpd.conf and everything will work.

You can start and stop it form /etc/init.d/init.sma and it is started from /etc/rc3.d/S82initsma.

Hopefully nobody has noticed, but I have upgraded the blog software to the latest and greatest version from wordpress. It was only about 9 or 10 version behind and 2 years old. If you have tried to go to the photo gallery, you have noticed that it currently isn’t working right now. I’m trying to find time to get that working again. I need to upgrade the gallery software and then upgrade the connection plugin to work with it. Hopefully I’ll have it done by next week. Summer is always the time when I have the least amount of time; Trying to finish the basement, clean the garage, get the yard respectable still, and jet skiing. I’m also working on getting my Solaris Systems certification, which I would like to have done before next year.

There was a Service Desk upgrade that I was involved in this weekend.  My job was to upgrade the Service Desk software on our Solaris servers.  The upgrade was simple enough, but when I went to upgrade the database I ran into issues.  To upgrade the database, you need to run a graphical GUI which requires the ability to forward X11 to the local machine through ssh.  I had checked and made sure my ssh client had that enabled and that Ming was running correctly.  I had to use Ming through windows because when I’m in linux the window doesn’t show up correctly.

When I went to upgrade the database, the program errored with the following message:

 Service Desk Message: Error: Cannot read the dictionary, can not find labels.rep and messages.rep, put this in \repo\locales\1033

I wasn’t able to find anything in google that could explain the issue, so I had to figure out what the cryptic message meant.  I had checked to see if the files were actually there or not and they were.  In the end, I determined that I forgot to make sure that sshd had X11 forwarding enabled.  Turned on X11 forwarding through sshd and the program ran correctly.  So in the end, that message basically meant that it couldn’t forward the X11 traffic to my laptop.

I had to do some development in perl that requied scrubbing a HTTPS page.  I ran into an issue when I moved the script from the laptop to the production solaris server that it was not able to process the HTTPS page.  Digging down a little deeper, I realized I never installed one of the required SSL modules for perl to allow libwww modules to connect to https pages.

You could actually use either Crypt::SSLeay or IO::Socket::SSL (which also required Net::SSLeay).  I decided to install them all to be on the safe side.  Net::SSLeay and IO::Socket::SSL had no problems, but when I tried to make Crypt::SSLeay, I ran into an issue.

When I tried to make the Makefile, I did the following:

root@appsrv /opt/downloads/cpan/Crypt-SSLeay-0.55 # perl Makefile.PL
Skipping testcover target, ExtUtils::MakeMaker::Coverage not found
=======================================================
Only one OpenSSL installation found at /usr/local/ssl
Consider running ‘perl Makefile.PL –default’ the next
time Crypt::SSLeay is upgraded to select this directory
automatically thereby avoiding the following prompt.
=======================================================
Which SSL install path do you want to use? [/usr/local/ssl]

BUILD INFORMATION
================================================
ssl library: OpenSSL 0.9.8 in /usr/local/ssl
ssl header:  openssl/ssl.h
libraries:   -L/usr/local/ssl/lib -lssl -lcrypto -lgcc
include dir: -I/usr/local/ssl/include/openssl
================================================
WARNING: LICENSE is not a known parameter.
‘LICENSE’ is not a known MakeMaker parameter name.
Writing Makefile for Crypt::SSLeay
The test suite can attempt to connect to public servers
to ensure that the code is working properly. If you are
behind a strict firewall or have no network connectivity,
these tests may fail (through no fault of the code).

Do you want to run the live tests (y/N) ? [N]
root@appsrv /opt/downloads/cpan/Crypt-SSLeay-0.55 # make
gcc -B/usr/ccs/bin/ -c  -I/usr/local/ssl/include/openssl -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O   -DVERSION=\"0.55\" -DXS_VERSION=\"0.55\" -fPIC "-I/usr/local/lib/perl5/5.8.7/sun4-solaris/CORE"   SSLeay.c
In file included from SSLeay.xs:25:
crypt_ssleay_version.h:1:25: openssl/ssl.h: No such file or directory
crypt_ssleay_version.h:2:28: openssl/crypto.h: No such file or directory
crypt_ssleay_version.h:3:25: openssl/err.h: No such file or directory
crypt_ssleay_version.h:4:26: openssl/rand.h: No such file or directory
crypt_ssleay_version.h:5:28: openssl/pkcs12.h: No such file or directory
SSLeay.xs:43: error: syntax error before ‘*’ token
SSLeay.xs: In function `InfoCallback’:
SSLeay.xs:48: error: `where’ undeclared (first use in this function)
SSLeay.xs:48: error: (Each undeclared identifier is reported only once
SSLeay.xs:48: error: for each function it appears in.)
SSLeay.xs:48: error: `SSL_ST_MASK’ undeclared (first use in this function)
SSLeay.xs:50: error: `SSL_ST_CONNECT’ undeclared (first use in this function)
SSLeay.xs:52: error: `SSL_ST_ACCEPT’ undeclared (first use in this function)
SSLeay.xs:57: error: `SSL_CB_LOOP’ undeclared (first use in this function)
SSLeay.xs:58: error: `s’ undeclared (first use in this function)
SSLeay.xs:60: error: `SSL_CB_ALERT’ undeclared (first use in this function)
SSLeay.xs:61: error: `SSL_CB_READ’ undeclared (first use in this function)
SSLeay.xs:63: error: `ret’ undeclared (first use in this function)
SSLeay.xs:66: error: `SSL_CB_EXIT’ undeclared (first use in this function)
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_new’:
SSLeay.c:118: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:118: error: `RETVAL’ undeclared (first use in this function)
SSLeay.xs:102: error: `ctx’ undeclared (first use in this function)
SSLeay.xs:135: error: `SSL_OP_ALL’ undeclared (first use in this function)
SSLeay.xs:137: error: `SSL_VERIFY_NONE’ undeclared (first use in this function)
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_free’:
SSLeay.c:172: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:172: error: `ctx’ undeclared (first use in this function)
SSLeay.c:176: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_set_cipher_list’:
SSLeay.c:193: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:193: error: `ctx’ undeclared (first use in this function)
SSLeay.c:200: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_use_certificate_file’:
SSLeay.c:218: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:218: error: `ctx’ undeclared (first use in this function)
SSLeay.c:226: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_use_PrivateKey_file’:
SSLeay.c:244: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:244: error: `ctx’ undeclared (first use in this function)
SSLeay.c:252: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_use_pkcs12_file’:
SSLeay.c:270: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:270: error: `ctx’ undeclared (first use in this function)
SSLeay.xs:170: error: `EVP_PKEY’ undeclared (first use in this function)
SSLeay.xs:170: error: `pkey’ undeclared (first use in this function)
SSLeay.xs:171: error: `X509′ undeclared (first use in this function)
SSLeay.xs:171: error: `cert’ undeclared (first use in this function)
SSLeay.xs:172: error: `ca’ undeclared (first use in this function)
SSLeay.xs:173: error: `PKCS12′ undeclared (first use in this function)
SSLeay.xs:173: error: `p12′ undeclared (first use in this function)
SSLeay.c:285: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_check_private_key’:
SSLeay.c:321: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:321: error: `ctx’ undeclared (first use in this function)
SSLeay.c:327: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__CTX_set_verify’:
SSLeay.c:345: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:345: error: `ctx’ undeclared (first use in this function)
SSLeay.c:354: error: syntax error before ‘)’ token
SSLeay.xs:212: error: `SSL_VERIFY_NONE’ undeclared (first use in this function)
SSLeay.xs:217: error: `SSL_VERIFY_PEER’ undeclared (first use in this function)
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_new’:
SSLeay.c:386: error: `SSL_CTX’ undeclared (first use in this function)
SSLeay.c:386: error: `ctx’ undeclared (first use in this function)
SSLeay.xs:231: error: `SSL’ undeclared (first use in this function)
SSLeay.xs:231: error: `ssl’ undeclared (first use in this function)
SSLeay.c:391: error: `RETVAL’ undeclared (first use in this function)
SSLeay.c:395: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_free’:
SSLeay.c:440: error: `SSL’ undeclared (first use in this function)
SSLeay.c:440: error: `ssl’ undeclared (first use in this function)
SSLeay.c:444: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_set_fd’:
SSLeay.c:461: error: `SSL’ undeclared (first use in this function)
SSLeay.c:461: error: `ssl’ undeclared (first use in this function)
SSLeay.c:468: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_connect’:
SSLeay.c:486: error: `SSL’ undeclared (first use in this function)
SSLeay.c:486: error: `ssl’ undeclared (first use in this function)
SSLeay.c:492: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_accept’:
SSLeay.c:510: error: `SSL’ undeclared (first use in this function)
SSLeay.c:510: error: `ssl’ undeclared (first use in this function)
SSLeay.c:516: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_write’:
SSLeay.c:534: error: `SSL’ undeclared (first use in this function)
SSLeay.c:534: error: `ssl’ undeclared (first use in this function)
SSLeay.c:546: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_read’:
SSLeay.c:590: error: `SSL’ undeclared (first use in this function)
SSLeay.c:590: error: `ssl’ undeclared (first use in this function)
SSLeay.c:603: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_get_peer_certificate’:
SSLeay.c:655: error: `SSL’ undeclared (first use in this function)
SSLeay.c:655: error: `ssl’ undeclared (first use in this function)
SSLeay.c:656: error: `X509′ undeclared (first use in this function)
SSLeay.c:656: error: `RETVAL’ undeclared (first use in this function)
SSLeay.c:660: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_get_verify_result’:
SSLeay.c:679: error: `SSL’ undeclared (first use in this function)
SSLeay.c:679: error: `ssl’ undeclared (first use in this function)
SSLeay.c:684: error: syntax error before ‘)’ token
SSLeay.xs:373: error: `X509_V_OK’ undeclared (first use in this function)
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_get_shared_ciphers’:
SSLeay.c:704: error: `SSL’ undeclared (first use in this function)
SSLeay.c:704: error: `ssl’ undeclared (first use in this function)
SSLeay.c:713: error: syntax error before ‘)’ token
SSLeay.xs:383: warning: assignment makes pointer from integer without a cast
SSLeay.c: In function `XS_Crypt__SSLeay__Conn_get_cipher’:
SSLeay.c:732: error: `SSL’ undeclared (first use in this function)
SSLeay.c:732: error: `ssl’ undeclared (first use in this function)
SSLeay.c:738: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__X509_free’:
SSLeay.c:757: error: `X509′ undeclared (first use in this function)
SSLeay.c:757: error: `cert’ undeclared (first use in this function)
SSLeay.c:761: error: syntax error before ‘)’ token
SSLeay.c: In function `XS_Crypt__SSLeay__X509_subject_name’:
SSLeay.c:778: error: `X509′ undeclared (first use in this function)
SSLeay.c:778: error: `cert’ undeclared (first use in this function)
SSLeay.c:786: error: syntax error before ‘)’ token
SSLeay.xs:407: warning: assignment makes pointer from integer without a cast
SSLeay.c: In function `XS_Crypt__SSLeay__X509_issuer_name’:
SSLeay.c:808: error: `X509′ undeclared (first use in this function)
SSLeay.c:808: error: `cert’ undeclared (first use in this function)
SSLeay.c:816: error: syntax error before ‘)’ token
SSLeay.xs:419: warning: assignment makes pointer from integer without a cast
SSLeay.c: In function `XS_Crypt__SSLeay__X509_get_notBeforeString’:
SSLeay.c:838: error: `X509′ undeclared (first use in this function)
SSLeay.c:838: error: `cert’ undeclared (first use in this function)
SSLeay.c:844: error: syntax error before ‘)’ token
SSLeay.xs:429: error: invalid type argument of `->’
SSLeay.c: In function `XS_Crypt__SSLeay__X509_get_notAfterString’:
SSLeay.c:863: error: `X509′ undeclared (first use in this function)
SSLeay.c:863: error: `cert’ undeclared (first use in this function)
SSLeay.c:869: error: syntax error before ‘)’ token
SSLeay.xs:437: error: invalid type argument of `->’
make: *** [SSLeay.o] Error 1
root@appsrv /opt/downloads/cpan/Crypt-SSLeay-0.55 #

I checked and made sure that the LD_LIBRARY_PATH had the openssl libraries in it, which it did.  I then went into the generated Makefile and changed line 147 from

INC = -I/usr/local/ssl/include/openssl

to

 INC = -I/usr/local/ssl/include

Which fixed the problem.  I was then able to make and install the module. 

You heard right, Google thinks that my ip address that I use at my home is located in the UK.  I know this by the pages google displays to me and the ads I see.  When I do a search on google, I get sent to UK google.  I can go back to the regular google, only to be sent back to the UK google.  The ads on google also display adds for people in the uk.  I got to other sites that use Google Adsense and they will only display adds for the UK.  It’s starting to annoy me.

I’m going through Current for my ISP.  From the looks of it, they are based out of New York.  The service is great and when I do traceroutes, it never goes near the UK.

I’ve spent a couple hours trying to find a way to let google know of the issue, but it’s close to impossible to find a place you can just send an email to let them know of the issue.  I actually had an easier time finding a snail mail address for technical support.  If anybody knows how to get this corrected, please let me know. 

To update the MySQL server,  you really do not have to run a patch.  That is a common question I was getting from people was where to get the patch.  You actually have to patch the OS and then from there run a command on MySQL.  If you’re OS doesn’t use zoneinfo, there is even a fix for this without having to reinstall a newer version of MySQL.

Taken from http://dev.mysql.com/doc/refman/5.0/en/time-zone-support.html:

If your system has its own zoneinfo database (the set of files describing time zones), you should use the mysql_tzinfo_to_sql program for filling the time zone tables. Examples of such systems are Linux, FreeBSD, Sun Solaris, and Mac OS X. One likely location for these files is the /usr/share/zoneinfo directory. If your system does not have a zoneinfo database, you can use the downloadable package described later in this section.

The mysql_tzinfo_to_sql program is used to load the time zone tables. On the command line, pass the zoneinfo directory pathname to mysql_tzinfo_to_sql and send the output into the mysql program. For example:

shell> mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql   

mysql_tzinfo_to_sql reads your system’s time zone files and generates SQL statements from them. mysql processes those statements to load the time zone tables.

mysql_tzinfo_to_sql also can be used to load a single time zone file or to generate leap second information:

• To load a single time zone file tz_file that corresponds to a time zone name tz_name, invoke mysql_tzinfo_to_sql like this:

  shell> mysql_tzinfo_to_sql tz_file tz_name | mysql -u root mysql 

  With this approach, you must must execute a separate command to load the time zone file for each named zone that the server needs to know about.

• If your time zone needs to account for leap seconds, initialize the leap second information like this, where tz_file is the name of your time zone file:

  shell> mysql_tzinfo_to_sql –leap tz_file | mysql -u root mysql 

If your system is one that has no zoneinfo database (for example, Windows or HP-UX), you can use the package of pre-built time zone tables that is available for download at the MySQL Developer Zone:

http://dev.mysql.com/downloads/timezones.html   

This time zone package contains .frm, .MYD, and .MYI files for the MyISAM time zone tables. These tables should be part of the mysql database, so you should place the files in the mysql subdirectory of your MySQL server’s data directory. The server should be stopped while you do this and restarted afterward.

Warning: Do not use the downloadable package if your system has a zoneinfo database. Use the mysql_tzinfo_to_sql utility instead. Otherwise, you may cause a difference in datetime handling between MySQL and other applications on your system.

Haven’t posted lately because I’ve been so busy.  The few times I have posted, I posted stuff on the family blog since this one is more technical.  If you’ve left a comment and it didn’t show up, that’s because my comment system has been getting blasted with spam.  I’ve tried my best to sort it out, but some probably got lost.  We’re currently working on our basement, so once that is done in a couple months, I’ll probably start working on my MAME cabinet.  Until later, have a good one.