On Page 124 of v3.5a User's Guide there is a description of how to increase the default Java heap space by changing the MAXHEAP variable in the file comsol.opts, which is loaded on launch. The default value that came with v3.5 was 256 MB. Try increasing this, taking into account how much memory you have in your machine. Let us know how this works.
More Pages to Explore .....