A journal to record my notes and ideas related to software development and computing

Tuesday, October 28, 2008

Finding large files

Yet another thing I keep forgetting:

find . -size +10000k

Finds files over roughly 10Mb

Friday, October 24, 2008

Proxy instantiation problem from Hibernate Session load

This one had me puzzled for a while yesterday. I was able to retrieve objects from a Session via a query, but I was getting the following error when I was using the load method:

org.hibernate.HibernateException: Javassist Enhancement failed: Thing
at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.getProxy(JavassistLazyInitializer.java:142)
at org.hibernate.proxy.pojo.javassist.JavassistProxyFactory.getProxy(JavassistProxyFactory.java:72)
at org.hibernate.tuple.entity.AbstractEntityTuplizer.createProxy(AbstractEntityTuplizer.java:402)
at org.hibernate.persister.entity.AbstractEntityPersister.createProxy(AbstractEntityPersister.java:3483)
at org.hibernate.event.def.DefaultLoadEventListener.createProxyIfNecessary(DefaultLoadEventListener.java:298)
at org.hibernate.event.def.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:219)
at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:126)
at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:905)
at org.hibernate.impl.SessionImpl.load(SessionImpl.java:822)
at org.hibernate.impl.SessionImpl.load(SessionImpl.java:815)
...
Caused by: java.lang.InstantiationException: Thing_$$_javassist_0
at java.lang.Class.newInstance0(Class.java:335)
at java.lang.Class.newInstance(Class.java:303)
at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.getProxy(JavassistLazyInitializer.java:139)
... 49 more

When I changed the code from

session.load(classOf[Thing], id).asInstanceOf[Thing]
to

session.createQuery("from Thing where id = " + id).list().asInstanceOf[java.util.List[Thing]].toList(1)
everything worked as expected.

I found a few results from the web, but nothing to indicate what my specific problem was.

The code I was working on was loosely based on another Scala project, which uses an older version of Hibernate. I changed the libraries over, and got a similar problem with CGLIB:

org.hibernate.HibernateException: CGLIB Enhancement failed: Thing
at org.hibernate.proxy.pojo.cglib.CGLIBLazyInitializer.getProxy(CGLIBLazyInitializer.java:96)
at org.hibernate.proxy.pojo.cglib.CGLIBProxyFactory.getProxy(CGLIBProxyFactory.java:49)
at org.hibernate.tuple.entity.AbstractEntityTuplizer.createProxy(AbstractEntityTuplizer.java:379)
at org.hibernate.persister.entity.AbstractEntityPersister.createProxy(AbstractEntityPersister.java:3455)
at org.hibernate.event.def.DefaultLoadEventListener.createProxyIfNecessary(DefaultLoadEventListener.java:257)
at org.hibernate.event.def.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:191)
at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:103)
at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:878)
at org.hibernate.impl.SessionImpl.load(SessionImpl.java:795)
at org.hibernate.impl.SessionImpl.load(SessionImpl.java:788)
...
Caused by: java.lang.InstantiationException: Thing$$EnhancerByCGLIB$$e29dbda1
at java.lang.Class.newInstance0(Class.java:335)
at java.lang.Class.newInstance(Class.java:303)
at org.hibernate.proxy.pojo.cglib.CGLIBLazyInitializer.getProxyInstance(CGLIBLazyInitializer.java:107)
at org.hibernate.proxy.pojo.cglib.CGLIBLazyInitializer.getProxy(CGLIBLazyInitializer.java:93)
... 49 more

This time I was luckier with my search results, which eventually lead me to these two pages. Indeed, the problem was due to having a private no-args constructor in my Thing object. As soon as I changed the constructor to be visable, the calls to load() started working as expected.

I was thrown a bit by the InstantiationException, which according to the JavaDoc is

Thrown when an application tries to create an instance of a class using the newInstance method in class Class, but the specified class object cannot be instantiated because it is an interface or is an abstract class.

While I can see that it's related, I don't really think the private constructor issue strictly falls under that description. In any case, I'm sure that a better message, or a different exception, or an update to the documentation, would have helped me isolate the problem a lot quicker.

Oddly, the code from the other project also has a private no-args constructor, yet it seems to work. That part remains a mystery to me, but I thought I'd post my other findings in case it helps anyone else who runs into a similar issue.

Thursday, October 16, 2008

Home directory version control notification

I spent some time in admin mode recently, seeing if I could improve the way my home directory is versioned. Previously, I had a few Mercurial repositories scattered around the place to track my documents, scripts, config and miscellaneous files.

Most of my work-related directories have their own Subversion repositories attached to them, but there were also some scratch projects that weren't under any VCS. My config files (eg, .bashrc) were sym-linked to a separate directory that had its own repository.

The main problems I had with the different repositories is that I didn't have an easy way to determine if I had any outstanding changes that need to be committed (my documents directory was shockingly out of date), and there were things that I wanted versioned (for example, application preferences) that weren't being versioned.

My first thought was to add a single Mercurial repository at the root of my home directory so I only had one repository to remember, and could easily track anything that gets added over time (for example, config files for new programs). Once I started thinking about all of the entries in my ignore file though, and how I wanted to share some stuff between work and home but not others, this option became less and less appealing.

So, back to the multiple repositories it was. What I ended up was repositories for the following directories:

~/Documents
~/Library
~/home
~/projects

The ~/Library directory is the standard directory on OSX for storing application support files, preferences, etc, and while I'm not particularly interested in the day-to-day happenings of this directory, it is helpful to have snap-shots that I can revert to at a later date.

I'm using the home repository to track utility scripts and dot files that live directly under ~ (mostly run config and history files). Almost all dot files in my ~ directory are sym-linked to somewhere in my home repository, and I have a bootstrap script to take care of this if I need to setup another home directory on a different machine in the future.

Finding out how to merge my old scripts and config repositories (whilst retaining their histories) was not a simple exercise. I thought I might be able to use Mercurial's import and export commands to merge one into the other, but there doesn't appear to be a way of specifying a relative path to import a repository at, so when I tried, I ended up with everything in the root of the repository. In hindsight, I realise I could have created a new repository and imported both repositories there, making sure to move the contents straight after importing, but there was another similar solution that I eventually found and got to work.

Now that I was happy with my repositories layout, I wanted a way to make sure they were kept up to date. At first I thought about adding a cron job that adds, removes, and commits changes automatically at given periods. Then I thought that might lead to stuff being committed that I didn't want (for example, if I generated a large amount of data that slipped through my ignore file), and having a repository log full of static messages. So instead, I decided to go with a notification approach instead.

Finding uncommitted changes was pretty straight forward with the following command:

find . -name .hg -not -path "./Library/.hg" -exec /opt/local/bin/hg st {}/.. \;

(I took the Library repository out of the results because I didn't want to be bugged about those changes throughout the day; instead, I just take a snapshot of that at the end of each day).

This has the added benefit that if I add more Mercurial repositories in the future, they'll automatically be included in the notification without me having to change anything.

Next up, notification. I find Growl on the Mac to be useful for receiving reasonably unobtrusive notifications throughout the day. It ships with a command-line interface (that you need to install separately), which was perfect for what I needed:

/usr/local/bin/growlnotify "Uncommitted Changes" <$CHANGES_FILE


$CHANGES_FILE holds the output of the previous find command. I wanted it stored so I could also copy the changes to my desktop, giving me a secondary visual cue that I have changes outstanding in case I missed the Growl notification.

I ended up with the following bit of script (after adding a few finishing touches, such as using a temp file for the intermediate results, only notifying if the size of the changes file is greater than zero, and displaying the notification with a nice Mercurial icon):

TMP_FILE=`mktemp /tmp/uncommitted_changes.XXXXXX` || exit 1

cd $HOME
find . -name .hg -not -path "./Library/.hg" \
-exec /opt/local/bin/hg st {}/.. \; >$TMP_FILE

if [[ -s $TMP_FILE ]]; then
CHANGES_FILE=$HOME/Desktop/uncommitted_changes.txt
mv $TMP_FILE $CHANGES_FILE
/usr/local/bin/growlnotify \
--image $HOME/bin/img/hg-icon-50.png \
"Uncommitted Changes" <$CHANGES_FILE > /dev/null 2>&1
else
rm $TMP_FILE
fi

And here it is in action:



One thing to note is that the output of growlnotify is being ignored. This is because of a known issue of invoking growlnotify from cron, whereby the notification still works but the following warning is mailed to you:

growlnotify[1592:10b] could not find local GrowlApplicationBridgePathway, falling back to NSDNC

I tried using the network settings described here, but this resulted in my notifications not showing up at all. For now, I'm just ignoring the output of growlnotify. I figure if something else goes wrong with it, it'll most likely result in notifications not showing up, which I think I will notice once I start getting used to them.

One final thing to mention is that I had intermittent trouble with using the --image flag when explicitly running the script. It sounds related to this problem, in that the notification would only be displayed intermittently but no other error or warning was emitted. Interestingly, I don't seem to get the problem when the script runs from cron, so I've left the flag in for now.

So there you go; notifications for uncommitted changes under my home directory, and a great reminder for me to go back and look at whether I want to keep configuration changes that are usually made when my mind is on other things. I've currently got it set to run every 15 minutes, which seems to be a reasonable balance between being too annoying and having too many outstanding changes.

Friday, October 3, 2008

Replacement in Bash parameter expansions

I have a few utility scripts to make working on the command line easier. One of them is a shortcut for temporarily getting a file or directory out of the way, by renaming it with a .bak suffix. So instead of typing
mv tmp/foo.txt tmp/foo.txt.bak
I can type

bk tmp/foo.txt
and achieve the same thing.

The script to do this was this simple one-liner
mv $1 "$1.bak"

Unfortunately, due to using tab-completion for file names, I'd often get the following error
bk tmp/bar/
mv: rename tmp/bar/ to tmp/bar/.bak: Invalid argument
which is caused by the trailing slash that is included when you tab-complete the directory name. So, ideally, what I wanted to do was check if there was a trailing slash and remove it for the second argument to mv.

Conveniently, this is easily achieved using a feature of Bash parameter expansions:

${parameter/pattern/string}
The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string. If pattern begins with ‘/’, all matches of pattern are replaced with string. Normally only the first match is replaced. If pattern begins with ‘#’, it must match at the beginning of the expanded value of parameter. If pattern begins with ‘%’, it must match at the end of the expanded value of parameter. If string is null, matches of pattern are deleted and the / following pattern may be omitted. If parameter is ‘@’ or ‘*’, the substitution operation is applied to each positional parameter in turn, and the expansion is the resultant list. If parameter is an array variable subscripted with ‘@’ or ‘*’, the substitution operation is applied to each member of the array in turn, and the expansion is the resultant list.

A quick change to my script and all is well:
mv $1 "${1/%\//}.bak"

Incidentally, calling it with the wrong directory (something like sudo bk /usr/lib) can be a bit disastrous, but that's a different story.

About Me

Computer programmer with and interest in music, and a passion for brewing beer, which I'm working at developing into a career!