You can set JAVA_HOME locally, in the command prompt. When you do that, it will only affect the current prompt, and nothing else on the machine.
To do so write, in the command prompt
set JAVA_HOME=xxx
You may find it easier to create a batch file to run your ant in jdk 1.4, instead of doing this every time. When you run a batch file the JAVA_HOME will change only for that execution. The batch file will look like this:
@echo off
set JAVA_HOME=xxx
ant ...