< Blog Home

Using Compressor with multiple cores

Here's a quick tip that a lot of people still don't know about. So you've just bought that shiny new Mac Pro, you finish your Final Cut Project, export to Compressor and wait.... and wait... and wait. Hang on a second, that brand-new 8-core machine is only utilizing one core!

Welcome to the world of Final Cut Studio, which is completely unaware of multiple cores in your system. Hopefully this will change in future versions but in the meantime, here is a trick to using those cores in Compressor.

1. Go to System Preferences and open the Apple Qmaster section.



2. In the Setup tab, click Stop Sharing if Qmaster is already activated.



2. Select QuickCluster if it is not already selected.

3. Under Services, select the service marked Compressor and select the checkboxes for Share and Managed.

3. Then click Options for Selected Service.

4. Now you need to select the number of instances to use for the cluster. This is the number of copies Qmaster will spawn to compress the job, and is a question that's open to debate.



Apple's official advice is that the number of instances should be half the number of cores. So if you have a quad-core machine, you should select two cores and four if you have an 8-core machine.

However, newer Mac Pros (early 2009 onwards) and MacBook Pros (early 2010 onwards) support a feature called HyperThreading which allows each core to act as two separate "virtual cores", allowing two tasks to run simultaneously on a single core. So for these models you should match the instances to the number of cores - i.e. four instances for a quad-core and eight for an 8-core.

Some people have suggested setting the instances to the total number of cores (or virtual cores on the newer machines) minus one. I find that this depends on the format you are compressing. If you are compressing a "light" format like DV, you could probably get away with it. However, more processor-intensive formats like H.264 will need a lot more headroom. You would also need at least 1 GB of RAM per instance.

Another point to bear in mind is that Compressor needs to stitch the separate pieces of the file together once processing is complete. More instances mean more pieces to assemble, which adds to the time taken. Consequently, the time saved by increasing the number of cores may be offset by the increase in the time taken to assemble the final file. I would therefore recommend sticking with Apple's advice of half the number of cores for early Mac Pros and matching the number of cores for later models.

5. If you are using Shake. Maya or have any other command-line renderer set up with Qmaster, select the Rendering item in the list and click Options for Selected Service.

6. Click the + button and add as many "Local machines" as necessary (see step 4 for details), then click Ok.



7. Under the QuickCluster section, type a name for your new virtual cluster.

7. Click Start Sharing and close System Preferences.

8. In Final Cut Pro, export your sequence as a QuickTime movie. This is important. Do not choose File > Export > Compressor, as this will fail.

9. In Compressor, import the movie, set up your batch and click Submit.

10. Instead of "This machine", select the name of your new cluster in the Cluster list and click Submit. The movie clip will be processed by multiple cores in your machine.



This is called Virtual Clustering, and it is achieved by launching a new version of Compressor for every instance. This is much easier from the perspective of Compressor's programmers but it takes up more memory and is less efficient than if they were to implement true multi-core capabilities into the application.

So be aware that using this method will require considerably more memory than rendering on a single instance. Apple recommends 1 GB RAM per instance but again, this depends on the media you are transcoding to. If you set too many instances and don't have enough memory, your machine would slow down (or worse, crash) thus undoing the whole point of setting up the virtual clusters in the first place. For best results, follow the recommendations in Step 4.

See the article Speeding up Compressor for more tips on improving Compressor performance.

Update 2010-12-06: This forum thread has an interesting comparison between This Computer and a Quick Cluster with various instances. Results vary depending on the chosen codec but the fastest times seem to occur with roughly half the number of instances compared to cores. The data would also suggest that a Quick Cluster is more beneficial on long files than short ones.