The acid test – a 360 x 180 QuickTime panorama.

In Part II I explained how to calibrate your lens/camera combination for use with PTMac, using just three heavily overlapped images.

While that article addressed use of the Canon EOS 5D and 15mm Fisheye, the approach is similar for any camera/lens combination. One caveat. If you are using a less wide lens, be sure to take the pictures so that you have sufficient overlap that a detail in the middle of the center picture appears in the left and right hand snaps also, and not too close to the edges of the outlying two images. This will make it possible to establish control points not just between adjacent images but between non-adjacent ones as well, heightening the accuracy of the calibration. Once calibrated this ‘large overlap’ approach is not needed with real life panoramas.

1 – Taking the pictures:

The deliverable is a 360 x 180 panorama, meaning a complete circle around the scene, with no breaks, and the ability to pan up to the sky and down to the ground with no ‘holes’ in the image.

For the 5D + Canon Fisheye, which has a horizontal angle of view of 91.7 degrees when mounted vertically, this dictates that six horizontal pictures be taken – theoretically you could get away with just four (4 x 91.7 > 360, if you get my drift) but that would result in such small overlap between images that the chances of a properly stitched panorama are zero. Likewise, five images might do, but control points would be too close to the edges of adjacent images, which PTMac does not like, so we go with six images. That gives a 53% overlap, if you must know, which falls under the description of ‘generous but not wasteful’ – everything your Government is not.

That means 8 clicks on the KingPano head between pictures, each click being 7.5 degrees. As the clicks are not very firm I double check my work by looking at the degree scale – starting at 0, the next five pictures are at 60, 120, 180, 240 and 300 degrees. All of this, of course, after very carefully aligning the first (or ‘anchor’) image with the accessory shoe mounted double bubble level. Now the bubbles do not stay perfectly centered as you rotate, as King Pano recognizes in its instruction sheet, but you can easily readjust the camera to dead level in each shot with the three thumb wheel adjusters in the KingPano’s base. It takes seconds to do.

Those will complete the circle but will leave holes at the top and bottom. So after returning the KingPano head to its original position, the 0 setting, loosen the big knob on top of the vertical plate and rotate the camera 90 degrees so that the lens points up to the sky. Take a picture. Loosen the knob again and point the camera down to the ground – take another picture. This one will, of course, include the tripod and KingPano in the field of view. These are known as the ‘zenith’ and ‘nadir’ images and require special processing.

Here are the six ‘circle’ images:

And here are the nadir and zenith images:

2 – Process the images:

First load the images into PTMac – load only the six images constituting the horizontal circle. We will not be using PTMac for stitching of the zenith and nadir images as the program is simply too clunky to allow that to be easily done. We will plug them in later using Photoshop, ImageAlign and CubicConverter.

Then on the Lens Settings tab, load the file you created in the calibration process:

Go to the Image Parameters tab and click on Arrange, select (Shift-Click in the left hand column) Images 0-5 and make the entries shown below:

This table reflect the fact that the horizontal images have to be turned 90 degrees CCW and the angle between each is 60 degrees.

Unlike during the calibration process where Control Points were manually entered, we will tell PTMac to automate the process. Go to the Preferences->SIFT (what the hell does that mean?) ans set ‘Number of Control Points: to 5, as here:

Go to the Control Points tab and click on ‘Auto match’ at the bottom:

PTMac will generate 5 control points for each image pair and will report back with a message something like this when it’s done (the count will be zero through 5 in this case):

Go to the Optimizer tab and select images as shown:

We are leaving the Yaw, Pitch and Roll of Image 0 unchanged – the camera was carefully levelled – allowing the keying of Images 1-5 off this anchor image.

Go back to the Control Points tab and click on Table at the lower left. You will see a table of all Control Points which you can sort in descending distance order by clicking on the ‘Distance’ column heading.

Delete the worst (biggest number) from each image pair, leaving four control points per image pair. Re run the Optimizer.

Now we need to confer the calibrated field of view and ‘b’ parameter on the optimization process. Click on FOV and run the optimization. Then click on ‘b’ and run the optimization again.

Go to the Create Panorama tab, select Enblend TIFF [.tif] for File format and click on ‘Create Panorama:’

3 – Photoshop check-up:

Load the image into Photoshop and it looks like this:

Now from the Filter menu load the Panagea plug-in and you can do a proper QuickTime preview of the image, like this:

As you can see, I have once more involved Bertie the Border Terrier in the action, but I’m afraid you cannot buy him anywhere, even at B&H, and he’s not for sale.

This preview shows the image is almost perfect, but any troublesome areas can be fixed by alternating between the preview and PTMac (Command-Tab) and adding or fixing control points in troublesome areas. The panorama previewer in PTMac (Command-E) is simply not up to the task, I’m afraid, showing overlap errors where there are none.

4 – Adding the nadir and zenith images:

While PTMac can add zenith and nadir images, once you add them and regenerate control points your nice low control point distances for the circle images get shot, and there’s no way to just generate control points for the zenith and nadir images alone – PTMac insists on redoing all Control Points. I constantly got messed up circle image stitches using this approach. As the holes left at top and bottom with the fisheye lens are very small, it’s far easier to simply defish the zenith and nadir images using the ImageAlign plug-in (see my earlier pieces on this) and insert them via Cubic Converter and Photoshop. More about how to do this appears here.

6 – Adjusting the tonal range:

Reload the final panorama into Photoshop, check for alignment and, if all is well make adjustments to Levels, Curves, tonal range as you like. If stitching is still suspect, repeat the fine tuning process above. Save the TIFF file. Then save your work file in PTMac (File->Save As) – this creates a file with the .ptm extensionj which saves all references to your images and the optimization settings. Useful if you ever want to go back and fine tune things.

7 – The QuickTime conversion:

It now remains to convert the TIFF file into a QuickTime movie. I use Cubic Converter to do this – it’s far more flexible than the straightforward function in PTMac, allowing you to dictate the starting point of the panorama when it is first loaded (mine, of course, starts with Bertie in the middle of the picture) as well as permitting constraints to be placed on the (default) huge zoom range which otherwise allows zooming in well past the resolution limits of the original.

Here is the file loaded into Cubic Converter:

Before saving it in the default Cubic QuickTime VR Movie format, I adjusted the starting point to center on Bert by moving the slider below the image. Click on Convert and Cubic Converter commences asembling the six cube faces, graphically displaying each as it does so on the cube shown.

When it’s done the display gives you Save options as so:

I elected Min/Max zoom settings of 35/70, Fast Start Grey preview (this displays grey in unloaded sections as the image loads) and the default size of 1200 x 600 pixels which is a nice fit for most screens.

If there are no zenith or naditr images present, I restrict the ‘Tilt’ setting to -65 to +65 degrees. that way the viewer will not be allowed to tilt so much that the ‘holes’ become visible.

The TIFF file is 46 mB in size, whereas the QuickTime movie is just 1.3 mB, and you can view it by clicking here. The aggregate size of the eight constituent images, each shot at the 5D’s lowest JPG setting to yield individual images sized 2496 x 1664 pixels, was some ten megabytes. Thus you can see that the QuickTime format is exceptionally economical, and the definition of the movie lacks nothing if you don’t go berserk zooming in – and I have prevented you from doing that with the Cubic Converter settings I chose.

And yes, feel free to zoom in on that all round dynamo and breed standard, Bertram the Border Terrier. He kept my morale high during the black dog days working my way through the labyrinthine software that is PTMac. As for the KingPano head, I have nothing but praise. It’s a third of the cost of the Manfrotto and it does the job, even with the heavy Canon 5D on board, thanks to the built in levellers. Well done Mr. King.

A note on processing time:

Now that everything is in order, I did a dry run to determine the processing time for a newly taken six picture 360 degree circular panorama, without zenith and nadir images. After taking the pictures, here’s how it went:

Load images into PTMac, auto-generate control points, run optimizer and generate panorama – first pass: 8 minutes.
Examine result in Photoshop using the Panagea plug-in and add control points as needed (6 added): 5 minutes
Regenrate control points and new panorama – second pass: 6 minutes.

The result is perfect.

Total processing time for six 1 mB files on a PPC 2 gHz, 2gB iMac: 19 minutes.

Now that’s what I call a good return on investment of time!

You think that’s long? Well, consider this. In 19 minutes Ansel Adams was still trying to remember where he left that bottle of pyro developer which he mistook for wine the other evening and when he did find it, what has he got three hours of dodging and burning later? Lung cancer and a lousy black and white print of some damned old rock in Yosemite. Gimme a break.

Three days of hard labor = one perfectly calibrated camera + lens.

There’s little use in setting up the King Pano head correctly if your camera/lens combination is incorrectly calibrated for the PTMac application.

PTMac comes with tons of lens/camera database settings which you can just load, but my experience suggests that you really should calibrate your particular body + lens, as calibration will adjust for your combination’s anomalies as well as for the offset of the sensor in the camera body. I cannot stress enough how important it is to do this, and as I have learned from my mistakes, you should be able to do this in a couple of hours by following the instructions below. This took me three days….

What follows is specific to the Canon EOS 5D + Canon 15mm EF Fisheye lens, but adapting these instructions to your camera/lens combination is a simple matter. It doesn’t help that the one set of lens parameters I found on the web for this combination (it does not exists in the PTMac database) did not work for my particular setup.

The problem with PTMac is that this great application is hampered by confusing instructions. The vendor is clearly very conscientious in responding to queries, but the result is that the magic sauce is in several places – the application’s Help file, the vendor’s forum and various FAQs also on the vendor’s web site, not to mention all sorts of chatter on Google. What follows is self contained – everything you need.

1 – Taking the calibration pictures

With the camera mounted in the King Pano as shown here, go to an indoor setting with as many horizontal and vertical lines as possible. Indoors is preferred to outdoors as you have total control of your environment – no wind movement, constant light, etc. – and you will be much closer to your subject, which makes for more critical calibration and, chances are, the setting will be replete with strong horizontal and vertical lines. Except for geodesic dome types, most homes tend to be rectangular.

I have my 5D’s custom setting (‘C’ on the mode dial) set for ISO 200, 1/125, f/8, color balance set to sunlight (not AWB), JPG quality set to the lowest possible (you want small files to speed up PTMac processing), auto rotate Off and the lens set to manual focus. Set the focus between the 3′ and infinity marks, adjust the shutter speed for exposure, leaving the aperture at f/8 and take a trial picture to check everything is nice and bright and easy to see. One half stop overexposed does not hurt. Now set the KingPano to the zero degree click stop and orient the camera + tripod so that the camera points to the leftmost side of the scene.

You will be taking just three pictures – each three click-stops apart on the KingPano – meaning 22.5 degrees as each click stop is equivalent to 7.5 degrees. Now while the full frame Canon 15mm fisheye will cover a 360 degree circle in just 6 shots (60 degrees between shots), we are opting for a huge overlap in the interest of best calibration.

Now level the camera carefully using the accessory shoe-mounted double bubble level I illustrated here . You want the camera to be level, so disregard the bubble level on the King Pano which will give false readings owing to the slight sag of the assembly from the weight of the 5D. That’s not the manufacturer’s fault. To make the King Pano much sturdier would greatly increase bulk and weight.

Take three pictures spaced three clicks apart.

2 – Load the pictures into PTMac:

The screen looks as follows:

3 – Guesstimate the lens settings:

Go to the Lens Settings tab and set the Lens type to Full-frame Fisheye[Rectangular] and the Hor. Field of View to 91.733 degrees. This will change as we calibrate but that’s Canon’s data sheet spec for the fisheye.

Note the Focal length multiplier is set at 1.000000 as the 5D is a full frame camera. Other Canons may be 1.3x or 1.6x, Nikon DSLRs are 1.5x and so on. You know your number. Leave everything else unchanged.

4 – Fix orientation:

Now go to the Image Parameters tab and this is what you will see:

The left hand column of images shows the full frame shot, the next column how these images are aligned by PTMac. Two things are wrong. The images are turned through 90 degrees and all overlap exactly.

Click on Arrange at the bottom of the screen and adjust the variables as shown below:

What you are doing is telling the application to turn every image 90 degrees CCW and to make the spacing between each 22.5 degrees, which is what the spacing was when you took the pictures.

Your screen now looks like this:

The pictures in the ‘Warped’ column are now turned upright and offset from one another.

5 – Set preliminary Control Points:

Now go to the Control Points tab. A Control Point is nothing more than two indicators – one in each picture – telling PTMac that those points either constitute identical points on those images or are points between horizontal or vertical lines. We will establish vertical line control points first – these tell PTMac about the Fisheye’s distortion.

In the following picture one Vertical control point has been placed at the top left of the vertical door jamb in Picture 1 and the second, in Picture 2 where the door jamb is most bowed from fisheye distortion.

You must also click the entry in the ‘Align’ column in the data box at lower left and choose Vertical Line to tell PTMac this is a vertical line control point pair.

Now you have to add five more control points which are spot Control points – remember to elect the Spot choice (a circle with a dot) in the ‘Align’ entry for the first of these – PTMac remains at whatever Control Point type was previously selected. Once chosen, a spot Control Point will be somewhat confusingly shown as ‘H+V’ in the Align column. No one ever accused engineers or software writers of being good at ergonomics, the guys at Aperture excepted. This is how it looks after adding five spot Control Points between Images 0 and 1 – strongly defined image details have been chosen not too close to the edges. PTMac does a nice job of magnifying the Control Point cursor in the small panels below the pictures as you move the mouse, clicking first in one image then in the other:

Now click on the right arrow key to the right of the Image 0 drop down menu at the top left and Image 0 becomes Image 1 on the left. Image 1, on the right, becomes Image 2. All Control Points have disappeared as none have yet been established between this image pair. Repeat the process – one set of vertical Control Points and five spot Control Points:

Now in each pair of images establish a pair of Horizontal Control Points – the example below illustrates point 7 between Images 1 and 2, at the top of the picture. Remember to elect the Horizontal Line choice in the Align table entry, as has been done here:

Finally, chose Image 0 for the left hand pane and Image 2 for the right. Create at least one control point which is identically placed between these non-adjacent images. You can do this by virtue of the huge overlap the 22.5 degree spacing affords – common information will apear in each of the three pictures.

6 – First Optimizer run:

The first Control Point pass is done. Go to the Optimizer tab and check the ‘Yaw:’, ‘Pitch:’, ‘Roll’: and ‘Use Control Points of:’ boxes as shown below. You have to click well to the right of the word ‘Image0’ and so on to get the check mark (or ‘tick’ to Her Majesty’s subjects) to show :

Here’s what the terms mean. Imagine you are on the Titanic, doing your Kate Winslet thing, facing the prow (front) of the ship, your arms outstretched horizontally to your sides, pointing to the horizon on either side. When the ship Rolls, your arms go up and down relative to the horizons. When it Pitches, the prow goes up and down in your dead ahead line of vision. When it Yaws, the prow moves from side to side as you look forward dead ahead. Just don’t ask me about port and starboard.

Click on the Run Optimizer box at the lower right. When the program is done, click the Field of View entry in the Lens parameters: box and run the Optimizer again. Then click the ‘b’ entry and run the Optimizer. Then click ‘a’ and ‘c’ and run the Optimizer. Finally, click both ‘Hshift’ and ‘Vshift’ and run the Optimizer one last time. By running the optimization process in multiple steps you heighten the accuracy of the result, as the program learns about your lens’s distortions and the camera’s sensor offset – the latter a normal result of manufacturing tolerances (Canon is emphatically not Leica in this regard!). Baby (optimization) steps beat one giant leap anyday.

Now take a look at the Lens Settings tab – you will see something like this (your numbers will differ):

Reflecting your optimization, PTMac has entered the various parameters in the boxes for your camera/lens combination.

We are not done. The critical point is now reached where you must fine tune your control points to get them as accurate as possible.

7 – Save your work:

But first, so as not to lose anything, click on the Lens Database box and you will be presented with this screen:

Click on Lens Database again and click on your saved settings’ name and you will see the following:

8 – Fine tune your control points:

Now we need to examine Control Points, so go to the Control Points tab an click on Table at lower left. This is what you will see:

We want to get the amounts shown in the Distance column as low as possible – below 2.0 ideally.

Click Goto and PTMac will go to the the images with your control point pair. Click the box above each picture to choose 100% display – that’s the largest you can get and allows you to critically reposition control points with the benefit of greater screen detail. Here’s how it looks. You can pan around with the slider bars at the sides of the pictures or Option-Drag the mouse cursor to move the image. Click the padlock in between the image tops to decouple their movement.

You can reposition any control point by grabbing it with the mouse pointer and click-dragging it. I cannot stress enough how critical this step is to a proper calibration. Go back to the table, chose the next over 2.0 entry in the list (none change until you re-run the Optimizer) and repeat. Keep doing this until you have addressed all point control points with a distance reading over 2.0. By all means check your horizontal and vertical control points also, but beware that PTMac gives wild readings for these which are not meaningful to the fine tuning process. If they are correctly set – meaning exactly on their respective vertical or horizontal lines – leave them alone.

9 – Now fine tune your Control Points again:

Now repeat the Optimizer routine above, doing it in many steps as before. Check the Lens Settings tab and you will see the data have changed, so resave them as before:

Now go back to Control Points and again re-tune any spot control points over 2.0. Keep doing this until all are under 2.0. It takes time. Believe me, the pay back in time saved from a correctly calibrated camera/lens combination is worth it. Now resave the lens settings.

10 – Preview the panorama on the screen:

You can now preview the panorama on the screen with Command-E, but be warned that the previewer is very crude and does not properly show alignment. On no account should you even think of manually dragging the images in the previewer screen to secure better alignment. You will waste a lot of time and you will fail.

11 – Generate the panorama:

You must have Enblend (on the Kekus.com web site) installed in a directory on your hard drive and you must input the path to Enblend in the Preferences settings for PTMac, otherwise the panorama generation will fail.

Click on the Create Panorama tab and select Enblend TIFF in the drop down box. This is the only setting that results in the best possible blending of stitches between adjacent pictures. Leave everything else alone and click Create Panorama at the lower right.

The panorama creation process takes a while – with these three small files (each ~800k) on my 2 mHz, 2gB PPC iMac it took some three minutes – this is what you will see towards the end of the process:

12 – Get the dog out – sorry, I mean Photoshop:

Load the TIFF file generated by PTMac and Enblend into Photoshop and this is what you will see:

Note that sophisticated man about town, Bertram the Border Terrier, earning his modelling fee. A dog far superior to anything from Adobe/Macromedia. You can use other breeds, of course, but I find the Border Terrier works best.

13 – Install the PanoPreviewer plug-in:

To allow previewing of this in QuickTime format, go to Panagea’s web site and download and install the PanoPreviewer Photoshop Plug-in. Restart Photoshop, reload the image and you can preview the image in its full QuickTime glory by selecting the plug-in from Photoshop’s Filter menu. This is how it looks:

14 – Check for skew and stitching errors:

Pan around and check the whole thing. If your picture is skewed to the horizontal you need to recheck your camera levelling and do this all again. There are no short cuts. If you see any stitching errors, go back and rework the control points. If that still fails, recheck your KingPano setup, especially checking nodal point and alignment settings. The subject above is very, very challenging with a multitude of strong overlapping details very close to the lens. The door is but eight feet from the camera, despite appearances. If it works with this sort of subject, it will work anywhere!

Here are the final Control Point distance statistics (identical to the finally saved data in the lens database on the Lens Settings tab) after all this calibration:

Why PTMac has determined the Field of View (FoV) to be 134.9 degrees when the vertical FoV of the Canon Fisheye is 91.733 degrees beats me but, hey, it works!

You can see that the 5D’s sensor is offset some 15 and 12 pixels from dead center (the Hshift and Vshift numbers). Yours will be different which is why you should calibrate your camera/lens and not depend on someone else’s settings from a table. It may be that full frame sensor cameras with fisheyes are just more sensitive, I don’t know, but it is the only approach which worked for me. The result is simply perfect – the joins are impossible to see.

In Part III I will look at creating a full 360 degree panorama using six images at 60 degree angles to one another plus a nadir and zenith image. Right now, if you scroll up or down you will see holes where image data are missing, like so:

By the way, here are my final camera/lens calibrations – these are unique to this particular camera and lens; substituting another 5D body (I should be so lucky!) will not do, as the sensor offset will be different owing to manufacturing tolerances:

15 – View the result:

Click here to view the QuickTime movie created from the three images stitched above. The file is 600kB. As you will see, it’s only a partial panorama as images were only taken over a narrow range of Yaw.

There. I finally used a technical word! You can take the lad out of engineering, but you cannot take the engineer out of the lad.

You can pan with the mouse cursor in all directions and zoom in and out with the Shift and Control keys. You will need QuickTime on your computer – all Macs come with it installed.

16 – Buy an Apple computer is you don’t have one:

I believe there is even a version of QuickTime available for download from Apple for those poor unfortunates who have yet to see the light – they are called Windows users – but you need a Mac to use PTMac, so now’s the time to switch to one of these great machines. Each lock-up on your Windows machine costs you 30 minutes. Say your time is worth $50/hour. A new 17″ iMac with the maximum 2 gB of memory installed costs $1,600. So that’s 64 lock-ups or two months’ of use of Windows. Get a grip!