Bug 1042

Summary: Add autodetection of image type in TextureIO
Product: [JogAmp] Jogl Reporter: Sven Gothel <sgothel>
Component: coreAssignee: Julien Gouesse <gouessej>
Status: RESOLVED FIXED    
Severity: enhancement CC: gouessej, sgothel
Priority: ---    
Version: 2.3.2   
Hardware: All   
OS: all   
Type: FEATURE SCM Refs:
354f239a461ea2f5ddd494810252216014f9b90b 0608e185b3147c28d29b02af58205e639adf3f4e b0169344a9fc01c4fcc9a9c1a750815f02bc5633 2689c311cd0fea40740c7d9db68a4e40020b1281 a836175ecf20aaf71cd53f1cb79c6f68a2f87e2a 3e8ef0ae4305fede0f1ddac2fee476c76c5a25a3 c4ed57f617117e3e38319f1a44a0d066f1a332b3
Workaround: ---

Description Sven Gothel 2014-08-04 19:54:49 CEST
Julien Gouesse Bug 982 Comment 11:

+++
The real bug has been fixed in Ardor3D:
https://github.com/gouessej/Ardor3D/blob/master/ardor3d-jogl/src/main/java/com/ardor3d/image/util/jogl/JoglImageLoader.java#L148

Sven, please tell me whether this kind of thing should be moved into JOGL.

My bug report is invalid as JOGL's TGA loader was already able to load the default image used in Ardor3D since the very beginning. Ardor3D AWT image loader was the only loader able to "guess" the image format until I fixed that in com.ardor3d.image.util.jogl.JoglImageLoader.
+++
Comment 1 Sven Gothel 2014-10-01 17:12:56 CEST
(In reply to comment #0)
> Julien Gouesse Bug 982 Comment 11:
> 
> +++
> The real bug has been fixed in Ardor3D:
> https://github.com/gouessej/Ardor3D/blob/master/ardor3d-jogl/src/main/java/
> com/ardor3d/image/util/jogl/JoglImageLoader.java#L148
> 
> Sven, please tell me whether this kind of thing should be moved into JOGL.
> 
> My bug report is invalid as JOGL's TGA loader was already able to load the
> default image used in Ardor3D since the very beginning. Ardor3D AWT image
> loader was the only loader able to "guess" the image format until I fixed
> that in com.ardor3d.image.util.jogl.JoglImageLoader.
> +++

It would be great, if you can add references to each of 
the detected file formats [within the branch].

I.e. references to the file format spec/description,
or at least spell the names out.
Comment 2 Julien Gouesse 2014-10-03 10:49:38 CEST
(In reply to comment #1)
> 
> It would be great, if you can add references to each of 
> the detected file formats [within the branch].
> 
> I.e. references to the file format spec/description,
> or at least spell the names out.

Great suggestion. I can do it at least for most "open" formats.
Comment 3 Julien Gouesse 2015-03-31 15:03:16 CEST
JPEG:
      - JPEG: FF D8
      - JPEG/JFIF: 4A 46 49 46 (i.e the ascii code of "JFIF")
      - JPEG/EXIF: 45 78 69 66 (i.e the ascii code of "EXIF")

I haven't found any free of charge specification.

PNG:  
      - 89 50 4E 47 0d 0A 1A 0A (50 4E 47 => "PNG")
http://www.libpng.org/pub/png/spec/1.1/PNG-Rationale.html#R.PNG-file-signature

      N.B: 0x89 = 137 (base 10) and 137 - 256 = -119

GIF:
      - GIF87a
      - GIF89a
http://www.w3.org/Graphics/GIF/spec-gif87a.txt
http://www.w3.org/Graphics/GIF/spec-gif89a.txt

BMP:
      - 0x42 0x4D (i.e the ascii code of "BM")
http://www.fileformat.info/format/bmp/spec/e27073c25463436f8a64fa789c886d9c/view.htm
"Specifies the type of file. This member must be BM."

TIFF:
      - 49 49 (little endian)
      - 4D 4D (big endian)
http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf
"Bytes 0-1: The byte order used within the file. Legal values are:
“II” (4949.H)
“MM” (4D4D.H)"

      N.B: My source code seems to be a bit wrong.
Comment 4 Julien Gouesse 2015-03-31 15:28:30 CEST
216 - 256 = -40
216 (base 10) = D8 (base 16)
255 - 256 = -1
255 (base 10) = FF (base 16)
Then, the second test of the JPEG format is unnecessary.

It seems to be safer to look for FF D8 FF according to these documents:
http://www.faqs.org/faqs/jpeg-faq/part1/
http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=54989
Comment 5 Julien Gouesse 2015-03-31 15:44:50 CEST
PPM: 
      - PPM: "P6", i.e 0x50 0x36
      - plain PPM: "P3" i.e 0x50 0x33
http://netpbm.sourceforge.net/doc/ppm.html

PGM:
      - PGM: "P5", i.e 0x50 0x35
      - plain PGM: "P2" i.e 0x50 0x32
http://netpbm.sourceforge.net/doc/pgm.html

PBM:
      - PBM: "P4", i.e 0x50 0x34
      - plain PBM: "P1" i.e 0x50 0x31
http://netpbm.sourceforge.net/doc/pbm.html
Comment 6 Julien Gouesse 2015-04-01 20:28:05 CEST
I've updated the code: https://github.com/gouessej/Ardor3D/commit/119885c08bfc98724f683eff8c1c3c9924263394
Comment 7 Julien Gouesse 2015-08-06 13:11:41 CEST
I'll implement this mechanism in com.jogamp.opengl.util.texture.ImageIOUtil.getFileSuffix(InputStream stream) and com.jogamp.opengl.util.texture.ImageIOUtil.getFileSuffix(byte[] bytes). Sven will probably improve it by using a bit stream to make it work with byte buffers too.
Comment 8 Julien Gouesse 2015-08-06 14:01:31 CEST
SGIImage.isSGIImage(), DDSImage.isDDSImage() and similar methods will become redundant.

All classes extending TextureIO.StreamBasedTextureProvider (i.e our texture providers) will use ImageIOUtil.getFileSuffix().

I could detect the image file format earlier but it would supersede a similar mechanism already implemented in Java ImageIO and used by the AWT provider IIOTextureProvider which is still used in TextureIO when AWT is available.

Sven suggested to put the pattern used during the detection into a separate inner class.
Comment 9 Julien Gouesse 2015-08-09 17:28:21 CEST
This is the very first version, I still have to take Sven's suggestions into account: https://github.com/gouessej/jogl/commit/354f239a461ea2f5ddd494810252216014f9b90b
Comment 10 Julien Gouesse 2015-08-09 18:02:26 CEST
I have documented my code and I have called it in TextureIO. I have deprecated the redundant methods:
https://github.com/gouessej/jogl/commit/0608e185b3147c28d29b02af58205e639adf3f4e
https://github.com/gouessej/jogl/commit/b0169344a9fc01c4fcc9a9c1a750815f02bc5633
https://github.com/gouessej/jogl/commit/2689c311cd0fea40740c7d9db68a4e40020b1281

However, there is something wrong with the DDS and the SGI formats, the legacy code and the new one seem to read the bytes in different orders, maybe my code is wrong.
Comment 11 Sven Gothel 2015-08-18 03:53:33 CEST
commit 354f239a461ea2f5ddd494810252216014f9b90b
    Detection of the image format, first version (work in progress)

commit 0608e185b3147c28d29b02af58205e639adf3f4e
    Documents ImageIOUtil, see the bug report 1042

commit b0169344a9fc01c4fcc9a9c1a750815f02bc5633
    Calls ImageIOUtil in TextureIO to detect the image format of a stream

commit 2689c311cd0fea40740c7d9db68a4e40020b1281
    Deprecates SGIImage.isSGIImage() and DDSImage.isDDSImage()
Comment 12 Sven Gothel 2015-08-18 03:54:03 CEST
TODO (for myself): Review!
Comment 13 Julien Gouesse 2015-08-18 10:37:46 CEST
I did some checks on several files with http://en.webhex.net/ to be sure that I was looking at the right bytes, especially for the DDS and SGI formats, it should work.

Feel free to use a bit stream and to move the patterns into some internal classes. As long as I can use some input streams and some byte arrays, it's ok for me.
Comment 14 Sven Gothel 2015-08-19 05:59:27 CEST
commit a836175ecf20aaf71cd53f1cb79c6f68a2f87e2a
    ImageIOUtil: Left-Align branches for readability


commit 3e8ef0ae4305fede0f1ddac2fee476c76c5a25a3
    ImageIOUtil: Use unique public static final const String identifiers


commit c4ed57f617117e3e38319f1a44a0d066f1a332b3
    ImageIOUtil -> ImageType + ImageType.Util ; Fix implementation and test.
    
    - ImageIOUtil -> ImageType + ImageType.Util
    
    - ImageType.Util.getFileSuffix(..):
      - Fix byte type conversion, i.e. 'b == (byte)0x89',
        cast is required to avoid byte -> int conversion.
        Note: signed byte -128 - +128
    
      - Parse in O(1), i.e. lexicographical parsing
    
      - FIXME: We seem to have at least three type collisions, validate!
    
    - ImageType:
      - Complete T_* w/ API doc -> FIXME/TODO missing type references!
      - ImageType instancing via InputStream or manual type definition.
    
    - TextureData
      - Contains optional source ImageType
    
    - TextureProvider:
      - Deprecate newTextureData(..) variants other than InputStream
        simplifying TextureIO.
    
    - TextureProvider.SupportsImageTypes:
      - Added interface, allowing mapping ImageType -> provider
    
    - Tested standalone ImageType (TestImageTypeNEWT) and
      via TextureIO (TestTextureIONEWT) utilizing list of all
      test data (ImageTstFiles), i.e. PNG, JPG, TGA and DDS.
Comment 15 Julien Gouesse 2015-08-21 11:07:16 CEST
The missing information:

BMP bitmap image file https://msdn.microsoft.com/en-us/library/dd183386%28VS.85%29.aspx

DIB device independent bitmap

DCX derivated of PCX

3d2 Stereo CAD-3D 2

3dmf 3D Meta File

CAM QV-10 Camera File

CBD Vector Map Data Format

CE2 Computer Eyes File

COB Caligari Truespace 2 File

CVG Calamus Vector Graphics

DEM Vista Landscape Format

DRW Micrographx Graphic

FTS Flexible image Transport System

GRO HP-48/49 GROB

HDR ArcoInfo Binary Image

IMG GEM Raster file

INFINI-D Infini-D Graphics File

IWC WaveL Image

J6I Ricoh Camera Image File

JIF Jeff's Image Format

KDC Kodak Camera DC20/40/50

L64 64LAN Image File

LBM Interchange File

LDF (Algo Vision LuraTech) LuraDocument Format

LWF LuraWave Format

MBM Psion Series 5 Bitmap

MGL MosASCII Graphics Library File

MPW MosASCII Project Workspace File

MSP Microsoft Windows Paint File

N64 64NET Image File

NCR NCR G4

NFF WorldToolKit Neutral File Format

NGG Nokia Group Graphics

NLM Nokia Logo File

NOL Nokia Operator Logo

PAT Gimp Pattern

PAX Secure Image File

PCD Kodak PhotoCD

PCL Page Control Language

PCX ZSoft PC Paintbrush

PIC Softimage

PIX PABX Background

POL Polygon Model File

QFX Fax Image File

QTM Apple Quick Time File

RIX ColoRIX File

SID Seamless Image Graphic File

SLD Slide Library File

SOD Storm 3D Object Definition

WIC J Wavelet Image Codec

WLM CompW Image

WMF Windows MetaFile https://msdn.microsoft.com/en-us/library/cc250370.aspx

WPG WordPerfect Graphic

XBM X11 Bitmap