Nope! One of the features of zip files (when the format was made) is that they could (potentially) be spread across multiple floppy disks, and if you wanted to add or remove a file from the zip, you only needed to edit the last floppy disk (which contained the central directory headers, and the end of central directory header). This means you could have "junk" files at the beginning of the zip file, but since they're not referenced the the central directory, they shouldn't be unziped.
Also, the magic numbers that >>2504
describes is the first part of the signature for a local header and end of central directory. It's entirely possible to have a zip file with no files inside it, in which case the magic number will be 0x06054b50. If there's at least one file, it'll start with the magic number for a local header, 0x04034b50.
The way zlib and the windows unzip utility works is by looking at the end of the file, and scanning backwards until they find the magic number for the end of central directory header, then they get the central directory headers, and finally the local headers and the associated compressed files.>>2541
That just appends a zip file to a png, it's the same as a binary concat; your offsets will be wrong.