Creating Page Layout Preview Images

Images are located in en-us/PreviewImages from the Master Page Library. Edit a current PNG image in the en-us/en-us/Preview Images Directory (I edited BlankWebPartPage to get a starting one). Upload to this directory, check-in and approve

On your parent site go to Site Actions –> Site Settings –> Master Pages and page Layouts under Galleries

You can associate a custom Template with an image by adding the address of the png image to the Preview Image Fields in the Edit Page Properties.
http:///_catalogs/masterpage/en-US/Preview%20Images/.png

/_catalogs/masterpage/en-US/Preview Images/.png

Make sure you change the http to https if you are on a secure site (or you will get the unsecure pop-up which is very annoying)