Eric's Notes: jasig

Showing posts with label jasig. Show all posts

Wednesday, May 22, 2013

Fixing ICS Time Zone

I was getting my agenda setup for the upcoming 2013 Apereo conference in San Diego and was excited to see their scheduling app let you select presentations and export a .ics file. Yay, something easy to import into Google Calendar and I'll have my whole schedule right on my phone!

Well, that was the case until I imported the events and had no attached time-zone information ... boo!

So here is the quick-and-dirty hack to add time-zone information to your .ics file.

Add a VTIMEZONE Block

These go right above the first BEGIN:VEVENT block in your ics file. You only need to add time-zone definitions for the time zones you want to reference in your events. At the bottom of the post are more timezone blocks

BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE

Update DTSTART and DTEND

For each VEVENT you want to set a timezone for update the DTSTART and DTEND to include a timezone reference.
Before:

DTSTART:20130602T130000
DTEND:20130602T160000

After:

DTSTART;TZID=America/Los_Angeles:20130602T130000
DTEND;TZID=America/Los_Angeles:20130602T160000

US VTIMEZONE List

Below are VTIMEZONE blocks for: Pacific, Mountain, Mountain - Arizona, Central, and Eastern time zones.

BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE

BEGIN:VTIMEZONE
TZID:America/Phoenix
X-LIC-LOCATION:America/Phoenix
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0700
TZNAME:MST
DTSTART:19700101T000000
END:STANDARD
END:VTIMEZONE

BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE

BEGIN:VTIMEZONE
TZID:America/New_York
X-LIC-LOCATION:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE

Monday, July 14, 2008

Unit testing Spring-LDAP code with ApacheDS

The Spring LDAP framework is a great library to assist in working with LDAP servers through a more manageable API. It provides the niceties that the DataSource interface does for JDBC along with support similar to the Spring JDBC framework. One area that has been lacking though is testing. With JDBC based code it is easy enough to use HSQLDB to provide an in-memory database to test against. With LDAP based code there hasn't been a good solution, until the Apache Directory project came along.

Apache Directory is a 100% Java LDAP implementation and includes support for in-memory LDAP instances for unit testing. Using this support plus a few helper classes to bridge the gap to Spring LDAP providing unit tests for LDAP related code is easy.

First add the necessary dependencies. If you are using Maven your project needs to depend on the following for testing:

<dependency> <groupid>junit</groupid> <artifactid>junit</artifactid> <version>4.4</version> </dependency> <dependency> <groupid>org.apache.directory.server</groupid> <artifactid>apacheds-core</artifactid> <version>1.0.2</version> </dependency> <dependency> <groupid>org.apache.directory.server</groupid> <artifactid>apacheds-server-jndi</artifactid> <version>1.0.2</version> </dependency> <dependency> <groupid>org.apache.directory.server</groupid> <artifactid>apacheds-server-unit</artifactid> <version>1.0.2</version> </dependency> <dependency> <groupid>org.slf4j</groupid> <artifactid>slf4j-log4j12</artifactid> <version>1.5.2</version> </dependency> <dependency> <groupid>org.springframework</groupid> <artifactid>spring-core</artifactid> <version>2.5.5</version> </dependency> <dependency> <groupid>org.springframework.ldap</groupid> <artifactid>spring-ldap</artifactid> <version>1.2.1</version> </dependency>

Second, add AbstractDirContextTest.java and SingleContextSource.java to your project.

Finally extend AbstractDirContextTest for your LDAP code test class.

package org.jasig.services.persondir.support.ldap; import java.util.Collections; import java.util.HashMap; import java.util.List; import java.util.Map; import org.jasig.services.persondir.util.Util; import org.springframework.core.io.ClassPathResource; import org.springframework.core.io.Resource; import org.springframework.ldap.test.AbstractDirContextTest; public class LdapPersonAttributeDaoTest extends AbstractDirContextTest { @Override protected String getPartitionName() { return "personDirectory"; } @Override protected Resource[] initializationData() { final ClassPathResource ldapPersonInfo = new ClassPathResource("/ldapPersonInfo.ldif"); return new Resource[] { ldapPersonInfo }; } public void testSingleAttrQuery() throws Exception { LdapPersonAttributeDao impl = new LdapPersonAttributeDao(); Map<String, Object> ldapAttribsToPortalAttribs = new HashMap<String, Object>(); ldapAttribsToPortalAttribs.put("mail", "email"); impl.setResultAttributeMapping(ldapAttribsToPortalAttribs); impl.setContextSource(this.getContextSource()); impl.setQueryAttributeMapping(Collections.singletonMap("uid", null)); impl.afterPropertiesSet(); Map<String, List<Object>> queryMap = new HashMap<String, List<Object>>(); queryMap.put("uid", Util.list("edalquist")); Map<String, List<Object>> attribs = impl.getMultivaluedUserAttributes(queryMap); assertEquals(Util.list("eric.dalquist@example.com"), attribs.get("email")); } }

Your test class needs to provide a partition name, which is the root of the DN for the server to test against, and a Resource array of ldiff files to load into the LDAP server. For the test above the ldiff file looke like:

Saturday, June 7, 2008

Apple's JDK6 Broke My Scripting!

uPortal 3 uses the new (JDK5+) Java scripting APIs via Cernunnos, the scripting language used in the data import/export scripts. This has been working great and is a huge step forward for the uPortal project. I've been building uPortal 3 on my Apple computer since development started and it has been working great.

Until I installed the Apple JDK6 Update.

Now I've been itching for JDK6 on my Mac for a while. I have little interest in the OS integration Apple does with the JDKs since I don't use it to develop code for OS X. I develop server-side web applications, all I need is the base JDK. So finally, around a year after Sun released JDK6, Apple gets their version out. I eagerly install the update, Eclipse still works, Tomcat starts and uP3 runs, we're all happy.

A few days later I grab the latest updates from the uPortal trunk and go to re-build and re-initialize the portal's database. It fails importing the data .... didn't this just work at the office? Didn't this just work here a few days ago?

Wondering if this is a new issue or not I grab the 3.0.0 release package and run initportal, it fails there too! All my attempts to run any of the Cernunnos scripts fail with the following stack trace:

[java] Exception in thread "main" java.lang.RuntimeException: Error parsing Main Grammar. [java] at org.danann.cernunnos.runtime.XmlGrammar.getMainGrammar(XmlGrammar.java:82) [java] at org.danann.cernunnos.runtime.ScriptRunner.<init>(ScriptRunner.java:63) [java] at org.danann.cernunnos.runtime.Main.main(Main.java:58) [java] Caused by: java.lang.RuntimeException: Unable to create the specified task: org.danann.cernunnos.runtime.EncloseGrammarTask [java] at org.danann.cernunnos.runtime.XmlGrammar.newTask(XmlGrammar.java:114) [java] at org.danann.cernunnos.runtime.ScriptRunner.compileTask(ScriptRunner.java:125) [java] at org.danann.cernunnos.runtime.XmlGrammar.getMainGrammar(XmlGrammar.java:74) [java] ... 2 more [java] Caused by: java.lang.RuntimeException: Unable to parse the specified entry: [java] name=groovy [java] impl=org.danann.cernunnos.script.ScriptPhrase [java] at org.danann.cernunnos.runtime.DefineEntryTask.perform(DefineEntryTask.java:141) [java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:58) [java] at org.danann.cernunnos.runtime.EncloseGrammarTask.init(EncloseGrammarTask.java:79) [java] at org.danann.cernunnos.runtime.XmlGrammar.newTask(XmlGrammar.java:110) [java] ... 4 more [java] Caused by: java.lang.RuntimeException: Error preparing the specified entry: org.danann.cernunnos.script.ScriptEnginePhrase [java] at org.danann.cernunnos.runtime.XmlGrammar.getEntry(XmlGrammar.java:343) [java] at org.danann.cernunnos.runtime.XmlGrammar.getEntry(XmlGrammar.java:321) [java] at org.danann.cernunnos.runtime.XmlGrammar.newPhrase(XmlGrammar.java:187) [java] at org.danann.cernunnos.ReagentType$1.evaluate(ReagentType.java:62) [java] at org.danann.cernunnos.runtime.DefineEntryTask.perform(DefineEntryTask.java:112) [java] ... 7 more [java] Caused by: java.lang.UnsupportedClassVersionError: Bad version number in .class file [java] at java.lang.ClassLoader.defineClass1(Native Method) [java] at java.lang.ClassLoader.defineClass(ClassLoader.java:675) [java] at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124) [java] at java.net.URLClassLoader.defineClass(URLClassLoader.java:260) [java] at java.net.URLClassLoader.access$100(URLClassLoader.java:56) [java] at java.net.URLClassLoader$1.run(URLClassLoader.java:195) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at java.net.URLClassLoader.findClass(URLClassLoader.java:188) [java] at java.lang.ClassLoader.loadClass(ClassLoader.java:316) [java] at java.lang.ClassLoader.loadClass(ClassLoader.java:306) [java] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280) [java] at java.lang.ClassLoader.loadClass(ClassLoader.java:251) [java] at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374) [java] at java.lang.Class.forName0(Native Method) [java] at java.lang.Class.forName(Class.java:242) [java] at sun.misc.Service$LazyIterator.next(Service.java:271) [java] at javax.script.ScriptEngineManager.initEngines(ScriptEngineManager.java:109) [java] at javax.script.ScriptEngineManager.access$000(ScriptEngineManager.java:37) [java] at javax.script.ScriptEngineManager$1.run(ScriptEngineManager.java:80) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at javax.script.ScriptEngineManager.init(ScriptEngineManager.java:78) [java] at javax.script.ScriptEngineManager.<init>(ScriptEngineManager.java:51) [java] at org.danann.cernunnos.script.ScriptEnginePhrase.<init>(ScriptEnginePhrase.java:38) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) [java] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) [java] at java.lang.reflect.Constructor.newInstance(Constructor.java:494) [java] at java.lang.Class.newInstance0(Class.java:350) [java] at java.lang.Class.newInstance(Class.java:303) [java] at org.danann.cernunnos.runtime.XmlGrammar.getEntry(XmlGrammar.java:326) [java] ... 11 more

So the next question is what the heck does uP3 depend on that is compiled to the JDK6 .class file format? Of course the UnsupportedClassVersionError is useless since it doesn't tell you what class file is bad or where it is loading it from!

The next step is adding the -verbose:class option to the Ant script that runs the Cernunnos scripts. This makes the JVM print out each class and the file it is loaded from, thats a lot of text but the only way to track this down. Re-running the script with the verbose option yields the culprit:

apple.applescript.AppleScriptEngineFactory

What the heck? Apple includes a scripting library for JDK6 only in the classpath for JDK5? They do this by default after installing their JDK6 update? This is not cool. The class is in the /System/Library/Java/Extensions/AppleScriptEngine.jar file which is added in the system's JDK extensions folder, apparently ALL Apple JDKs load ALL the jars from this directory. That makes it a bad place for a .jar that isn't compatible with all the JDKs on the system.

The solution I ended up coming up with is to move the AppleScriptEngine.jar into the JDK6 extensions directory:

This move fixed the problem and all is happy (mostly) in the land of Java on the Mac.

Wednesday, March 26, 2008

Java SQLException Chaining

In JDK 1.4 chained Throwables were added and made logging and tracing exceptions much easier. Being able to create a new exception and specify the causing exception in a standard way, tied to JVM printing out the stack chain, was a big boost to tracking down problems in an application.

There are still some exceptions that do not use this standard chaining mechanism and one of the most frustraiting cases in SQLException. It provides a getNextException() method which returns the next SQLException related to the failed operation. The problem here is without special handling code these nextExceptions just get lost in log messages.

A general solution to this problem is to add an Aspect that either logs the results of getNextException or adds them to the exception chain.

/** * Copyright 2007 The JA-SIG Collaborative. All rights reserved. * See license distributed with this file and * available online at http://www.uportal.org/license.html */ package org.jasig.portal.portlet.dao.jpa; import java.sql.SQLException; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.springframework.core.Ordered; /** * Provides additional logging for SQL based exceptions that provide chained exceptions via * {@link SQLException#getNextException()} * * @author Eric Dalquist * @version $Revision$ */ public class SQLNextExceptionLoggerAspect implements Ordered { protected final Log logger = LogFactory.getLog(this.getClass()); private int order = 0; /* (non-Javadoc) * @see org.springframework.core.Ordered#getOrder() */ public int getOrder() { return this.order; } /** * @param order the order to set */ public void setOrder(int order) { this.order = order; } public void logBatchUpdateExceptions(Throwable t) { while (t != null && !(t instanceof SQLException)) { t = t.getCause(); } if (t instanceof SQLException) { SQLException sqle = (SQLException)t; //If the SQLException is the root chain the results of getNextException as initCauses if (sqle.getCause() == null) { SQLException nextException; while ((nextException = sqle.getNextException()) != null) { sqle.initCause(nextException); sqle = nextException; } } //The SQLException already has a cause so log the results of all getNextException calls else { while ((sqle = sqle.getNextException()) != null) { this.logger.error("Logging getNextException for root SQLException: " + t, sqle); } } } } }

This aspect class can be used via Spring's AOP support with the following context configuration:

Sunday, February 24, 2008

PostgreSQL Backups

While shell scripting is not my strong suit I put this together to do automatic backups of the Postgres database that backs the ja-sig.org services. The script will vacuum each database, do a full dump of each database, do a dump of each schema and do a global meta-data backup. All the dumps are compressed with bzip2 and date/time stamped. Logs and backups older than 7 days are removed.

The results get stored in a date-stamped directory and all actions are logged into a date/time stamped log file.

The script can be easily customized using the variables at the top of the file and should work on any PostgreSQL database.

#!/bin/bash # setup environment PG_PORT=5432 PG_HOME=/usr/postgres/8.2 PG_VAR=/postgres/8.2 PG_DATA=$PG_VAR/data PG_BACKUPS=$PG_VAR/backup PG_LOGS=$PG_VAR/logs PG_BIN=$PG_HOME/bin PG_DUMPALL=$PG_BIN/pg_dumpall PG_DUMP=$PG_BIN/pg_dump PG_PSQL=$PG_BIN/psql PG_VACUUMDB=$PG_BIN/vacuumdb PG_BACKUP=$PG_BACKUPS/`date "+%Y%m%d"` PG_LOG_FILE=$PG_LOGS/backup_`date "+%Y%m%d_%H%M%S"`.log NICE="nice -n 20" COMPRESS="bzip2 -9" DUMPALL_ARGS="-p $PG_PORT -c -g -o" DUMP_ARGS="-p $PG_PORT -c -o -F t" VACUUM_ARGS="-z -p $PG_PORT -d" PSQL_ARGS="-p $PG_PORT -U postgres -q -c" echo "`date`: Script $0 starting" >> $PG_LOG_FILE 2>&1 # Make the backup directory mkdir -p $PG_BACKUP >> $PG_LOG_FILE 2>&1 # Iterate over all databases databases=`$PG_PSQL $PSQL_ARGS "\l" 2>> $PG_LOG_FILE | sed -n 4,/\eof/p | grep -v rows\) | awk {'print $1'}` for database in $databases do echo "`date`: Vacuuming database '$database'" >> $PG_LOG_FILE 2>&1 $PG_VACUUMDB $VACUUM_ARGS $database >> $PG_LOG_FILE 2>&1 echo "`date`: Vacuuming of database '$database' complete" >> $PG_LOG_FILE 2>&1 PG_DUMP_DB_FILE=$PG_BACKUP/backup.`date "+%Y%m%d_%H%M%S"`.$database.tar.bz2 echo "`date`: Backing up database '$database' to '$PG_DUMP_DB_FILE'" >> $PG_LOG_FILE 2>&1 $NICE $PG_DUMP $DUMP_ARGS $database 2>> $PG_LOG_FILE | $NICE $COMPRESS > $PG_DUMP_DB_FILE 2>> $PG_LOG_FILE echo "`date`: Backing up database '$database' to '$PG_DUMP_DB_FILE' complete" >> $PG_LOG_FILE 2>&1 # iterate over all schemas in this database schemas=`$PG_PSQL $PSQL_ARGS "\dn" -d $database 2>> $PG_LOG_FILE | sed -n 4,/\eof/p | grep -v rows\) | awk {'print $1'}` for schema in $schemas do PG_DUMP_SCHM_FILE=$PG_BACKUP/backup.`date "+%Y%m%d_%H%M%S"`.$database.$schema.tar.bz2 echo "`date`: Backing up schema '$schema' of database '$database' to '$PG_DUMP_SCHM_FILE'" >> $PG_LOG_FILE 2>&1 $NICE $PG_DUMP $DUMP_ARGS -n $schema $database 2>> $PG_LOG_FILE | $NICE $COMPRESS > $PG_DUMP_SCHM_FILE 2>> $PG_LOG_FILE echo "`date`: Backing up schema '$schema' of database '$database' to '$PG_DUMP_SCHM_FILE' complete" >> $PG_LOG_FILE 2>&1 done done # Dump global meta-data PG_DUMPALL_FILE=$PG_BACKUP/backup.`date "+%Y%m%d_%H%M%S"`.global_metadata.sql.bz2 echo "`date`: Dumping database cluster meta-data to '$PG_DUMPALL_FILE'" >> $PG_LOG_FILE 2>&1 $PG_DUMPALL $DUMPALL_ARGS 2>> $PG_LOG_FILE | $NICE $COMPRESS > $PG_DUMPALL_FILE 2>> $PG_LOG_FILE echo "`date`: Dumping database cluster meta-data to '$PG_DUMPALL_FILE' complete" >> $PG_LOG_FILE 2>&1 echo "`date`: Removing old backups" >> $PG_LOG_FILE 2>&1 find $PG_BACKUPS -mtime +7 -exec rm -Rf {} \; echo "`date`: Removing old backups complete" >> $PG_LOG_FILE 2>&1 echo "`date`: Removing old logs" >> $PG_LOG_FILE 2>&1 find $PG_LOGS -mtime +7 -exec rm -Rf {} \; echo "`date`: Removing old logs complete" >> $PG_LOG_FILE 2>&1 echo "`date`: Script $0 complete" >> $PG_LOG_FILE 2>&1 exit 0

Tuesday, January 15, 2008

Using Ant with a Maven project

I recently had to take a Maven 2 WAR project and make it buildable on a system that did not have Maven available. Not being one to duplicate configuration I put together an Ant build script that, when Maven is available, can be used to pull the dependencies into the project file structure in an organized fashion and write out properties about the maven project for use by the script.

Currently the script provides support for doing the equivalent of 'mvn clean package' on a simple Maven managed WAR project.

<project name="Gartner SSO Portlet" basedir="." xmlns:artifact="urn:maven-artifact-ant">  <property name="mavenScopes" value="provided,runtime,compile,test" /> <property name="lib.basedir" value="${basedir}/lib" /> <property name="lib.dir.compile" value="${lib.basedir}/compile" /> <property name="lib.dir.provided" value="${lib.basedir}/provided" /> <property name="lib.dir.test" value="${lib.basedir}/test" /> <property name="lib.dir.runtime" value="${lib.basedir}/runtime" /> <property name="antBuildPropertiesFile" value="${lib.basedir}/build.properties" />  <property name="pom.build.resourcesDirectory" value="${basedir}/src/main/resources" /> <property name="pom.build.testResourcesDirectory" value="${basedir}/src/test/resources" /> <property name="pom.build.webappResourcesDirectory" value="${basedir}/src/main/webapp" /> <property name="dest.dir" value="${basedir}/dist" /> <target name="clean" description="Required by MyUW build process. Cleans all generated files."> <property file="${antBuildPropertiesFile}" /> <delete dir="${pom.build.directory}" /> <delete dir="${dest.dir}" /> </target> <target name="dist" description="Required by MyUW build process. Packages a fully build portlet WAR" depends="resource:resources, compiler:compile, jspc:compile, resource:testResources, compiler:testCompile, surefire:test, war:war"> <property file="${antBuildPropertiesFile}" /> <mkdir dir="${dest.dir}"/> <copy file="${pom.build.directory}/${pom.build.finalName}.war" todir="${dest.dir}" /> </target> <target name="updateAntBuild" description="Updates the dependencies used by the Ant only build, requires Maven be installed.">  <delete dir="${lib.basedir}" />  <artifact:pom file="${basedir}/pom.xml" id="project.pom" />  <length property="baseDir.length" string="${basedir}/"/> <property name="localDirRegEx" value=".{${baseDir.length}}(.*)" /> <propertyregex property="build.sourceDirectory" input="${project.pom.build.sourceDirectory}" regexp="${localDirRegEx}" select="\1" /> <propertyregex property="build.testSourceDirectory" input="${project.pom.build.testSourceDirectory}" regexp="${localDirRegEx}" select="\1" /> <propertyregex property="build.directory" input="${project.pom.build.directory}" regexp="${localDirRegEx}" select="\1" /> <propertyregex property="build.outputDirectory" input="${project.pom.build.outputDirectory}" regexp="${localDirRegEx}" select="\1" /> <propertyregex property="build.testOutputDirectory" input="${project.pom.build.testOutputDirectory}" regexp="${localDirRegEx}" select="\1" />  <mkdir dir="${lib.basedir}" /> <echo file="${antBuildPropertiesFile}"> pom.build.finalName=${project.pom.build.finalName} pom.build.sourceDirectory=$${basedir}/${build.sourceDirectory} pom.build.testSourceDirectory=$${basedir}/${build.testSourceDirectory} pom.build.directory=$${basedir}/${build.directory} pom.build.outputDirectory=$${basedir}/${build.outputDirectory} pom.build.testOutputDirectory=$${basedir}/${build.testOutputDirectory} </echo>  <for param="scope" list="${mavenScopes}" delimiter=","> <sequential>  <artifact:dependencies pathid="project.pom.@{scope}.classpath" verbose="false" usescope="@{scope}"> <pom refid="project.pom" /> </artifact:dependencies>  <property name="lib.dir.@{scope}" value="${lib.basedir}/@{scope}" />  <mkdir dir="${lib.dir.@{scope}}" />   <trycatch> <try> <for param="pathFile"> <path refid="project.pom.@{scope}.classpath" /> <sequential> <basename file="@{pathFile}" property="@{pathFile}.name"/>  <var name="includeFile" value="true" /> <var name="doneLooking" value="false" /> <for param="orderedScope" list="${mavenScopes}" delimiter=","> <sequential> <if> <and> <equals arg1="false" arg2="${doneLooking}" /> <not> <equals arg1="@{scope}" arg2="@{orderedScope}" /> </not> </and> <then> <if> <available file="${lib.basedir}/@{orderedScope}/${@{pathFile}.name}" /> <then> <var name="includeFile" value="false" /> <var name="doneLooking" value="true" /> </then> </if> </then> <else> <var name="doneLooking" value="true" /> </else> </if> </sequential> </for> <if> <equals arg1="true" arg2="${includeFile}" /> <then> <copy file="@{pathFile}" todir="${lib.dir.@{scope}}" /> </then> </if> </sequential> </for> </try> <catch /> </trycatch> </sequential> </for> </target>  <target name="resource:resources"> <property file="${antBuildPropertiesFile}" /> <mkdir dir="${pom.build.outputDirectory}" /> <copy todir="${pom.build.outputDirectory}"> <fileset dir="${pom.build.resourcesDirectory}" /> </copy> </target> <target name="compiler:compile"> <property file="${antBuildPropertiesFile}" /> <mkdir dir="${pom.build.outputDirectory}" /> <javac srcdir="${pom.build.sourceDirectory}" destdir="${pom.build.outputDirectory}"> <classpath> <fileset dir="${lib.dir.provided}" /> <fileset dir="${lib.dir.runtime}" /> <fileset dir="${lib.dir.compile}" /> </classpath> </javac> </target> <target name="jspc:compile"> <echo>Not Implemented</echo> </target> <target name="resource:testResources"> <property file="${antBuildPropertiesFile}" /> <mkdir dir="${pom.build.testOutputDirectory}" /> <copy todir="${pom.build.testOutputDirectory}"> <fileset dir="${pom.build.testResourcesDirectory}" /> </copy> </target> <target name="compiler:testCompile"> <property file="${antBuildPropertiesFile}" /> <mkdir dir="${pom.build.testOutputDirectory}" /> <javac srcdir="${pom.build.testSourceDirectory}" destdir="${pom.build.testOutputDirectory}"> <classpath> <fileset dir="${lib.dir.provided}" /> <fileset dir="${lib.dir.runtime}" /> <fileset dir="${lib.dir.compile}" /> <fileset dir="${lib.dir.test}" /> <pathelement location="${pom.build.outputDirectory}" /> </classpath> </javac> </target> <target name="surefire:test"> <property file="${antBuildPropertiesFile}" /> <property name="pom.build.test.reportsDirectory" value="${pom.build.directory}/surefire-reports" /> <mkdir dir="${pom.build.test.reportsDirectory}" /> <junit fork="true" showoutput="true" printsummary="yes"> <classpath> <fileset dir="${lib.dir.provided}" /> <fileset dir="${lib.dir.runtime}" /> <fileset dir="${lib.dir.compile}" /> <fileset dir="${lib.dir.test}" /> <pathelement location="${pom.build.outputDirectory}" /> <pathelement location="${pom.build.testOutputDirectory}" /> </classpath> <formatter type="plain" /> <batchtest fork="yes" todir="${pom.build.test.reportsDirectory}"> <fileset dir="${pom.build.testOutputDirectory}" /> </batchtest> </junit> </target> <target name="war:war"> <property file="${antBuildPropertiesFile}" /> <property name="pom.war.directory" value="${pom.build.directory}/${pom.build.finalName}" /> <mkdir dir="${pom.build.directory}/${pom.build.finalName}" /> <copy todir="${pom.war.directory}"> <fileset dir="${pom.build.webappResourcesDirectory}" /> </copy> <copy todir="${pom.war.directory}/WEB-INF/classes"> <fileset dir="${pom.build.outputDirectory}" /> </copy> <copy todir="${pom.war.directory}/WEB-INF/lib"> <fileset dir="${lib.dir.runtime}" /> </copy> <war destfile="${pom.war.directory}.war" webxml="${pom.war.directory}/WEB-INF/web.xml"> <fileset dir="${pom.war.directory}" /> </war> </target>  <taskdef resource="net/sf/antcontrib/antlib.xml"> <classpath> <pathelement location="${basedir}/bootstrap/ant-contrib-1.0b3.jar" /> </classpath> </taskdef> <typedef resource="org/apache/maven/artifact/ant/antlib.xml" uri="urn:maven-artifact-ant"> <classpath> <pathelement location="${basedir}/bootstrap/maven-ant-tasks-2.0.7.jar" /> </classpath> </typedef> </project>

As you can see at the end of the script it depends on Ant Contrib and Maven Ant Tasks.

Wednesday, December 12, 2007

Spring 2.5 Module Contents

The 2.5 release of the Spring Framework re-organized many packages and classes in the specific module JARs. Moving from Spring 2.0 to 2.5 introduces some pain in figuring out what moved where and I couldn't find anything detailing which packages reside in which module JARs. So after a few lines of shell-fu I produced the following to help with the upgrades:

spring-jms.jar


META-INF/
org/
org/springframework/
org/springframework/jms/
org/springframework/jms/config/
org/springframework/jms/connection/
org/springframework/jms/core/
org/springframework/jms/core/support/
org/springframework/jms/listener/
org/springframework/jms/listener/adapter/
org/springframework/jms/listener/endpoint/
org/springframework/jms/listener/serversession/
org/springframework/jms/remoting/
org/springframework/jms/support/
org/springframework/jms/support/converter/
org/springframework/jms/support/destination/

spring-core.jar


META-INF/
org/
org/springframework/
org/springframework/asm/
org/springframework/asm/commons/
org/springframework/asm/signature/
org/springframework/core/
org/springframework/core/annotation/
org/springframework/core/enums/
org/springframework/core/io/
org/springframework/core/io/support/
org/springframework/core/style/
org/springframework/core/task/
org/springframework/core/task/support/
org/springframework/core/type/
org/springframework/core/type/classreading/
org/springframework/core/type/filter/
org/springframework/metadata/
org/springframework/metadata/commons/
org/springframework/util/
org/springframework/util/comparator/
org/springframework/util/xml/

spring-web.jar


META-INF/
org/
org/springframework/
org/springframework/remoting/
org/springframework/remoting/caucho/
org/springframework/remoting/httpinvoker/
org/springframework/remoting/jaxrpc/
org/springframework/remoting/jaxrpc/support/
org/springframework/remoting/jaxws/
org/springframework/web/
org/springframework/web/bind/
org/springframework/web/bind/annotation/
org/springframework/web/bind/support/
org/springframework/web/context/
org/springframework/web/context/request/
org/springframework/web/context/support/
org/springframework/web/filter/
org/springframework/web/jsf/
org/springframework/web/jsf/el/
org/springframework/web/multipart/
org/springframework/web/multipart/commons/
org/springframework/web/multipart/support/
org/springframework/web/util/

spring-aop.jar


META-INF/
org/
org/springframework/
org/springframework/aop/
org/springframework/aop/aspectj/
org/springframework/aop/aspectj/annotation/
org/springframework/aop/aspectj/autoproxy/
org/springframework/aop/config/
org/springframework/aop/framework/
org/springframework/aop/framework/adapter/
org/springframework/aop/framework/autoproxy/
org/springframework/aop/framework/autoproxy/target/
org/springframework/aop/interceptor/
org/springframework/aop/scope/
org/springframework/aop/support/
org/springframework/aop/support/annotation/
org/springframework/aop/target/
org/springframework/aop/target/dynamic/

spring-webmvc-portlet.jar


META-INF/
org/
org/springframework/
org/springframework/web/
org/springframework/web/portlet/
org/springframework/web/portlet/bind/
org/springframework/web/portlet/context/
org/springframework/web/portlet/handler/
org/springframework/web/portlet/multipart/
org/springframework/web/portlet/mvc/
org/springframework/web/portlet/mvc/annotation/
org/springframework/web/portlet/util/

spring-tx.jar


META-INF/
org/
org/springframework/
org/springframework/dao/
org/springframework/dao/annotation/
org/springframework/dao/support/
org/springframework/jca/
org/springframework/jca/cci/
org/springframework/jca/cci/connection/
org/springframework/jca/cci/core/
org/springframework/jca/cci/core/support/
org/springframework/jca/cci/object/
org/springframework/jca/context/
org/springframework/jca/endpoint/
org/springframework/jca/support/
org/springframework/jca/work/
org/springframework/transaction/
org/springframework/transaction/annotation/
org/springframework/transaction/config/
org/springframework/transaction/interceptor/
org/springframework/transaction/jta/
org/springframework/transaction/support/

spring-beans.jar


META-INF/
org/
org/springframework/
org/springframework/beans/
org/springframework/beans/annotation/
org/springframework/beans/factory/
org/springframework/beans/factory/access/
org/springframework/beans/factory/annotation/
org/springframework/beans/factory/config/
org/springframework/beans/factory/generic/
org/springframework/beans/factory/parsing/
org/springframework/beans/factory/serviceloader/
org/springframework/beans/factory/support/
org/springframework/beans/factory/wiring/
org/springframework/beans/factory/xml/
org/springframework/beans/propertyeditors/
org/springframework/beans/support/

spring-webmvc-struts.jar


META-INF/
org/
org/springframework/
org/springframework/web/
org/springframework/web/servlet/
org/springframework/web/servlet/view/
org/springframework/web/servlet/view/tiles/
org/springframework/web/struts/

spring-context-support.jar


META-INF/
org/
org/springframework/
org/springframework/cache/
org/springframework/cache/ehcache/
org/springframework/mail/
org/springframework/mail/javamail/
org/springframework/scheduling/
org/springframework/scheduling/commonj/
org/springframework/scheduling/quartz/
org/springframework/ui/
org/springframework/ui/freemarker/
org/springframework/ui/jasperreports/
org/springframework/ui/velocity/

spring-context.jar


META-INF/
org/
org/springframework/
org/springframework/context/
org/springframework/context/access/
org/springframework/context/annotation/
org/springframework/context/config/
org/springframework/context/event/
org/springframework/context/i18n/
org/springframework/context/support/
org/springframework/context/weaving/
org/springframework/ejb/
org/springframework/ejb/access/
org/springframework/ejb/config/
org/springframework/ejb/support/
org/springframework/instrument/
org/springframework/instrument/classloading/
org/springframework/instrument/classloading/glassfish/
org/springframework/instrument/classloading/oc4j/
org/springframework/instrument/classloading/weblogic/
org/springframework/jmx/
org/springframework/jmx/access/
org/springframework/jmx/export/
org/springframework/jmx/export/annotation/
org/springframework/jmx/export/assembler/
org/springframework/jmx/export/metadata/
org/springframework/jmx/export/naming/
org/springframework/jmx/export/notification/
org/springframework/jmx/support/
org/springframework/jndi/
org/springframework/jndi/support/
org/springframework/remoting/
org/springframework/remoting/rmi/
org/springframework/remoting/soap/
org/springframework/remoting/support/
org/springframework/scheduling/
org/springframework/scheduling/backportconcurrent/
org/springframework/scheduling/concurrent/
org/springframework/scheduling/support/
org/springframework/scheduling/timer/
org/springframework/scripting/
org/springframework/scripting/bsh/
org/springframework/scripting/config/
org/springframework/scripting/groovy/
org/springframework/scripting/jruby/
org/springframework/scripting/support/
org/springframework/stereotype/
org/springframework/ui/
org/springframework/ui/context/
org/springframework/ui/context/support/
org/springframework/validation/

spring-orm.jar


META-INF/
org/
org/springframework/
org/springframework/orm/
org/springframework/orm/hibernate3/
org/springframework/orm/hibernate3/annotation/
org/springframework/orm/hibernate3/support/
org/springframework/orm/ibatis/
org/springframework/orm/ibatis/support/
org/springframework/orm/jdo/
org/springframework/orm/jdo/support/
org/springframework/orm/jpa/
org/springframework/orm/jpa/persistenceunit/
org/springframework/orm/jpa/support/
org/springframework/orm/jpa/vendor/
org/springframework/orm/toplink/
org/springframework/orm/toplink/support/

spring-webmvc.jar


META-INF/
org/
org/springframework/
org/springframework/web/
org/springframework/web/servlet/
org/springframework/web/servlet/handler/
org/springframework/web/servlet/handler/metadata/
org/springframework/web/servlet/i18n/
org/springframework/web/servlet/mvc/
org/springframework/web/servlet/mvc/annotation/
org/springframework/web/servlet/mvc/multiaction/
org/springframework/web/servlet/mvc/support/
org/springframework/web/servlet/mvc/throwaway/
org/springframework/web/servlet/support/
org/springframework/web/servlet/tags/
org/springframework/web/servlet/tags/form/
org/springframework/web/servlet/theme/
org/springframework/web/servlet/view/
org/springframework/web/servlet/view/document/
org/springframework/web/servlet/view/freemarker/
org/springframework/web/servlet/view/jasperreports/
org/springframework/web/servlet/view/tiles2/
org/springframework/web/servlet/view/velocity/
org/springframework/web/servlet/view/xslt/

spring-test.jar


META-INF/
org/
org/springframework/
org/springframework/mock/
org/springframework/mock/jndi/
org/springframework/mock/web/
org/springframework/mock/web/portlet/
org/springframework/test/
org/springframework/test/annotation/
org/springframework/test/context/
org/springframework/test/context/junit38/
org/springframework/test/context/junit4/
org/springframework/test/context/support/
org/springframework/test/context/testng/
org/springframework/test/context/transaction/
org/springframework/test/jdbc/
org/springframework/test/jpa/
org/springframework/test/util/
org/springframework/test/web/

spring-jdbc.jar


META-INF/
org/
org/springframework/
org/springframework/jdbc/
org/springframework/jdbc/core/
org/springframework/jdbc/core/metadata/
org/springframework/jdbc/core/namedparam/
org/springframework/jdbc/core/simple/
org/springframework/jdbc/core/support/
org/springframework/jdbc/datasource/
org/springframework/jdbc/datasource/lookup/
org/springframework/jdbc/object/
org/springframework/jdbc/support/
org/springframework/jdbc/support/incrementer/
org/springframework/jdbc/support/lob/
org/springframework/jdbc/support/nativejdbc/
org/springframework/jdbc/support/rowset/

Tuesday, October 30, 2007

Poor Man's TimeMachine: rsync

The shell script on the computer being backed up, needs a no-password ssh key to work.


#!/bin/bash

BACKUP_HOST=user@host
SSH_KEY=/home/user/.ssh/host_rsa

LOG_DIR=~/logs
LOG_FILE=$LOG_DIR/backup.`date +%Y%m%d_%H%M`.log

SRC_DIR=/home/user/
DEST_BASE_DIR=/home/user/backups/localhost
DEST_DIR_NAME=backup_`date +%Y.%m.%d_%H.%M.%S`
PREV_DIR_NAME=previous

DEST_DIR=$DEST_BASE_DIR/$DEST_DIR_NAME
PREV_DIR=$DEST_BASE_DIR/$PREV_DIR_NAME

mkdir -p $LOG_DIR

nice -10 /usr/bin/rsync --rsh="ssh -i $SSH_KEY" \
 --checksum --archive --compress --human-readable --progress --recursive \
 --link-dest=$PREV_DIR \
 --filter='merge /home/user/backup/backup.filterrules' $SRC_DIR \
 $BACKUP_HOST:$DEST_DIR >> $LOG_FILE 2>&1

ssh -i $SSH_KEY $BACKUP_HOST "link $PREV_DIR $DEST_DIR" >> $LOG_FILE 2>&1

bzip2 $LOG_FILE

A Ruby script that I use with the password-less ssh key on the server side to reduce chances of it being abused. Just add command="~/bin/backup-wrapper.rb" before your key in .ssh/authorized_keys

#!/usr/bin/ruby SSH_ORIGINAL_COMMAND = ENV['SSH_ORIGINAL_COMMAND'] if SSH_ORIGINAL_COMMAND.nil? $stderr.puts "Environment variable SSH_ORIGINAL_COMMAND must be set" exit end # Check if the command is the rsync server rsyncCommand, prev_dir, dest_dir = /^rsync --server -[^ ;]+ --link-dest ([^ ;]+) \. ([^ ;]+)$/.match(SSH_ORIGINAL_COMMAND).to_a if !rsyncCommand.nil? dest_dir_base_idx = dest_dir.rindex('/') dest_dir_base = dest_dir[0..dest_dir_base_idx] dest_dir_name = dest_dir[(dest_dir_base_idx + 1)..dest_dir.length] # Make sure the parent of the destination directory exists mkdir_rslt = %x[mkdir -p #{dest_dir_base}] if !File.exist?(dest_dir_base) # turn execution over to the rsync command exec SSH_ORIGINAL_COMMAND else # check if the command is trying to change the link linkCommand, prev_dir, dest_dir = /^link ([^ ;]+) ([^ ;]+)$/.match(SSH_ORIGINAL_COMMAND).to_a if !linkCommand.nil? dest_dir_base_idx = dest_dir.rindex('/') dest_dir_base = dest_dir[0..dest_dir_base_idx] dest_dir_name = dest_dir[(dest_dir_base_idx + 1)..dest_dir.length] # Update the symlinks if File.exist?(dest_dir) # delete previous symlink if it exists rm_rslt = %x[rm #{prev_dir}] if File.exist?(prev_dir) # create new symlink to the latest destination directory ln_rslt = %x[ln -s #{dest_dir_name} #{prev_dir}] else $stderr.puts "Something went wrong, '#{dest_dir}' does not exist to link to. Previous link will not be touched." exit end else $stderr.puts "Illegal command passed to SSH '#{SSH_ORIGINAL_COMMAND}'" exit end end

I'm sure there is a way to do what the wrapper does in bash but I happen to be more comfortable with ruby doing the string parsing. If anyone wants to provide a bash script for doing the same thing I'd be more than happy to see it.