Texture compression

Hi joglers ^^

So, after some times I am more or less at that point I could start define somehow "stabil"

Here you go, jgli is the java port of gli

Many side features still miss, but the core is that, you can finally load DXGI and KTX texture with pure 100% (nub) java code!

Time to start gathering feedbacks...

Hi

It's better than nothing but why promoting a third party library instead of implementing those features directly within JOGL? Adding KTX and DXGI support in our texturing and imaging API would benefit to the whole community without having to learn how to use another API, a developer already using TextureIO would have nothing or almost nothing to modify in her/his code to support those formats.

Charity begins at home: I moved the automatic image format detection from JogAmp's Ardor3D Continuation into JOGL so that it benefits to all JOGL users instead of the few ones using this engine. This is what I expect from other developers.

Edit.: You have kept the useless comments generated by Netbeans at the beginning of each class... It would have been easier to support those image formats within JOGL...

Edit.2: Where is the license???

Edit.3: There is almost no comment in the source code and there are numerous calls to System.out whereas you should use a logger.

gouessej wrote

Hi

It's better than nothing but why promoting a third party library instead of implementing those features directly within JOGL? Adding KTX and DXGI support in our texturing and imaging API would benefit to the whole community without having to learn how to use another API, a developer already using TextureIO would have nothing or almost nothing to modify in her/his code to support those formats.

Charity begins at home: I moved the automatic image format detection from JogAmp's Ardor3D Continuation into JOGL so that it benefits to all JOGL users instead of the few ones using this engine. This is what I expect from other developers.

Edit.: You have kept the useless comments generated by Netbeans at the beginning of each class... It would have been easier to support those image formats within JOGL...

Edit.2: Where is the license???

Edit.3: There is almost no comment in the source code and there are numerous calls to System.out whereas you should use a logger.

What you write is totally right and agreedable.

And indeed it was also my first idea: I tried at the begin modifying the jogl structure to accomodate the dxgi formats (this was my attempt), but I found this way really time consuming and low productive. I spent literally days without getting nowhere...
The dds/dxgi docs are a nightmare, there is nothing given for sure, specifications look like just an hint, I spent a lot of days, I had something like 30 tabs opened on my browser, I was even looking inside other projects, like dds reader, Valve dx->gl porter and so, trying to understand what has to be translated with what...

The current structure of jogl was fine at that time, when you didn't have to deal with layer/faces, all the possible combination of 1D-2D-3D-Cube-Array and a legacy and a new format from M$... The main difference between the jogl current structure and jgli/gli is the usage of an intermediate layer, made of formats, targets, types, etc.. you can take a look at them here

The biggest problem is experience, I lack experience in the texture/opengl field and in developing for jogl. This was another reason to do 1:1 port.

1:1 port gives me essentially three huge advantages:

- I see progresses, coding porting a lib is way easier than doing it from zero. If anything doesn't work or I don't have a heck of idea how to go on, I just debug the C version and I quickly find it out what was the problem. Sometimes even this is not sufficient, look for example at my troubles with jAssimp and materials..

- I don't have to deal with all the shit of understanding and checking which parameters and in which order I have to control in order to figure the texture format out, basically I skipt the M$ specs :p

- I can incredibily easily and quick catch up the main lib, these two qualities also transfer in turn to other topics such as closing/finding bugs and introducing new features and formats (g-truc is proposing, for example, a new format, KMG). We basically exploit the C community for being always on the top edge ^^
Just to mention, I implemented (partially, only what has been ported so far) all the modifications that g-truc wrote in the whole month of September in barely an hour (!). And he is updating that like hell, trust me.


Anyway, I still plan to re-introduce all the features jogl has, such as png, jpeg and so on. Once done and stable, we can include it back in jogl.

Of course this will include some modifications/new methods and so on, but I am truly convinced the advantages are far bigger.

What do you think?


Ps 1: I'll remove then when it will be time, there are other priorities now, if you want to do, don't hesitate :)

Ps 2: Mit

Ps 3: same as 1

You could have used another library as a source of inspiration to make it work within JOGL but now, you have something that works, good.

If you don't, I'll find some time next year to add KTX support within JOGL. I still think that it isn't difficult to reintroduce g-truc's changes into another API.

What's the interest of open sourcing your code if it is hard to understand and lack of comments?

I don't like Microsoft but if you have no real understanding of its specifications, you'll be very dependent on g-truc's talent and you'll be unable to fix the bugs by yourself.

Your library is a Netbeans project, it depends on Netbeans, there is no clean Ant or Maven script. you should make an effort if you really want others to use your stuff. Look at my own code, it's very well commented, documented, quite easy to build, ...

I don't want to discourage you but the interest of supporting additional image formats in JOGL but through third party libraries is highly debatable and I'll do my best to drive them less and less appealing as time goes by. You could at least implement a SPI still in a separate library but loadable by JOGL at runtime.

gouessej wrote

You could have used another library as a source of inspiration to make it work within JOGL but now, you have something that works, good.

I took a brief look before taking the decision.

I ended up with GLI because:

- it was one of the very few supporting dxgi and ktx

- very active and with a good user base

- modern looking and oriented

- hosted on github

gouessej wrote

If you don't, I'll find some time next year to add KTX support within JOGL. I still think that it isn't difficult to reintroduce g-truc's changes into another API.

Ktx is not a problem, supporting both dds and dxgi is. Moreover supporting few formats is not a problem (jogl dds supports three compressed formats, two of them can be seen as one since they share the same block dimension). Supporting all of them is.

gouessej wrote

What's the interest of open sourcing your code if it is hard to understand and lack of comments?

You are right, but in relative small project like this one it matters relatively less. Just to say, a project like jBullet would be a different story. This does not mean I don't want to do it or I never will, I just think that, given all the contest conditions, I give low priority.
One of the advantage of open source is that if you think it has a high priority instead, you are very welcome to contribute, as I said, or in extreme case clone and do whatever you like.

gouessej wrote

I don't like Microsoft but if you have no real understanding of its specifications, you'll be very dependent on g-truc's talent and you'll be unable to fix the bugs by yourself.

Oh, but the specifications are clear, gouessej.. Let me paraphrase you what the problem is, scanning the specs you read many times: "this is how the specifications is, so you should write in this way, but for the reading part you should do this and this because everyone doesn't give a shit about specs".
Now, as I said, if you want to be able to support most of them out there, you need to have (also historical) experience in the dds/dxgi texture field, something I don't have.
You are right, this introduce g-truc dependency. But I think gli has a solid core and I already ported it, so fixing bugs will require relative less skillz than building a core by myself.

gouessej wrote

Your library is a Netbeans project, it depends on Netbeans, there is no clean Ant or Maven script. you should make an effort if you really want others to use your stuff. Look at my own code, it's very well commented, documented, quite easy to build, ...

You are definitely right, this is one of my major problem I keep postponing because of laziness, I'd try to keep it up with this asap.

gouessej wrote

I don't want to discourage you but the interest of supporting additional image formats in JOGL but through third party libraries is highly debatable and I'll do my best to drive them less and less appealing as time goes by. You could at least implement a SPI still in a separate library but loadable by JOGL at runtime.

I disagree, I cannot see any cons about having a larger user base.

I'll analyse the SPI option

Hi

Having a larger user base? For whom? For JGLI or for JOGL? This kind of feature has nothing to do in a third party library in my humble opinion. It forces a JOGL user to use another library instead of simply using TextureIO. If your library isn't good enough in terms of code quality, readability, maintainability, ..., it will be preferable to use something else in JOGL itself. You'll be forced to duplicate a bit some JOGL features at the end.

I speak mainly about KTX because I'm not interested in DXGI yet and because supporting KTX in JOGL seems to be doable to me. I'll detect KTX in my image file format detection mechanism first by using this file identifier:
0xAB, 0x4B, 0x54, 0x58, 0x20, 0x31, 0x31, 0xBB, 0x0D, 0x0A, 0x1A, 0x0A
https://www.khronos.org/opengles/sdk/tools/KTX/file_format_spec/#2.1

From gli, jgli and jogl.

Anyway TextureIO can stay there without problem, we can make in a way to change only the underlying hood so that for the end user nothing (or almost) will change

You wrote on JGO that you're planning to support PNG, JPG and GIF which is totally useless as the two first formats are already supported by JOGL and the last one is more or less covered by a patent.

Substituting the texture part under the hood includes keeping the support for what is already supported today... it seems pretty obvious

Thanks for the GIF, I didn't know that


Anyway, I am back at the texture package and I have the feeling the texture package would need some general cleaning..

Do we really still need for example:

- AWTTextureData
- AWTTextureIO
- TextureSequence
- TextureState

Regarding all the DDS/JPEG/SGI/TGA-Image, I see they share *a lot* of methods, I would propose we may have a single general class, called Image or TextureData as today and then extending it only when we really need some specific method for that format.

Separating the AWT-free classes from the classes depending on AWT is necessary as JOGL works under Android (=no AWT) and the compact profiles (since Java 1.8) don't contain AWT. Separating the classes that require a current OpenGL context from those that don't is important too, in order not to prevent some actions from being performed on a thread which isn't the thread dedicated to OpenGL operations. The texture sequence is used in the video API. In my humble opinion, the four classes you mentioned shouldn't be removed.

However, there is probably something to clean in the class named ***Image. The test case at the end of SGIImage should be removed. JPEGImage is already quite slim.

By the way, I strongly disagree with this argument:
"Another main advantage of gli over jogl, is that it uses internally an API-independent format (well actually much more, target, type, swizzle, etc), that acts also as a common medium when you need to handle texture from different APIs. This will give us much more flexibility to "build new floors", such as KMG, Vulkan, etc.."

There is already a very capable general purpose API to handle textures in JOGL and it's possible to write some SPIs based on third party libraries. It's viable. For example, you can write some code based on Apache Commons Imaging and call your methods in a SPI for JOGL.

I'm harsh but I agree with Sven, we want you to succeed.

gouessej wrote

There is already a very capable general purpose API to handle textures in JOGL and it's possible to write some SPIs based on third party libraries. It's viable. For example, you can write some code based on Apache Commons Imaging and call your methods in a SPI for JOGL.

I'm harsh but I agree with Sven, we want you to succeed.

I am sorry but it is not at all very capable nowadays. TextureData is basically an old texture2D-oriented class.

Let me bring some observations:

- it does not have depth (only width, height)

- it has an int to represent the internalFormat, an GLPixelAttributes object for the pixel type and the format amd a ColorSpace object. There is no clean way (sometimes just no way) to put all these entities in relationship between each other. You can't go from one to the other. Only internally in the GlPixelAttributes there is a small convertion to retrieve the internalFormat but that's all if I didn't miss anything else.

- it does not have any awareness of layers, faces and swizzle. This reflects on the missing of a method to retrieve the (sub)data of the texture at (layer, face, level).

In gli, I have a format for each API and the independent one.

The GL one holds internal format, external format, type and swizzle.
The neutral one is a very generic one and has block size, an array for the block dimension, number of components, swizzles and flags. This is compression option independent, that is it doesnt matter if the format is compressed or not, it is just a flag.

And gli texture data structure is deeply based on the layers, faces and levels suddivision. I can query any type of memory segment.

This offers a lot of flexibility to support all the different combination, for example loading an cube array with mipmap is as simple as creating three loops.

On 10/19/2015 01:27 PM, elect [via jogamp] wrote: Elect, now I am getting a bit out of patient here,
yes ofc - we have to enhance Texture and TextureData,
for crying out loud.

:)

yes .. as mentioned in at least one bug report for enhancement.

Again, I put your post into the category of 'whining',
just to be clear.

Also, if nothing will condense, well - I or Julien will have to make the
enhancement ourselves. But that is OK. Just tired of this discussion
and I would prefer to see actual work.

~Sven

Sven Gothel wrote

Elect, now I am getting a bit out of patient here,
yes ofc - we have to enhance Texture and TextureData,
for crying out loud.

:)

Don't tell me. It is quite a while since I am going repeating things that keep being misunderstood continuosly.

Sven Gothel wrote

Again, I put your post into the category of 'whining',
just to be clear.

Points of view. I put it into the "technic reply". Gouessej said one thing I thought it wasn't true and I tried to expose why. It's a normal discussion between two persons having different opinions.

Sven Gothel wrote

Also, if nothing will condense, well - I or Julien will have to make the
enhancement ourselves. But that is OK. Just tired of this discussion
and I would prefer to see actual work.

~Sven

I'm analyzing glPixelAttributes as you proposed..

Sven Gothel wrote

Also, if nothing will condense, well - I or Julien will have to make the
enhancement ourselves. But that is OK. Just tired of this discussion
and I would prefer to see actual work.

I have to update my repository. I can commit at least a few changes on the image format detection. I have to look at those bug entries to see what I can do.

Which bug reports are we talking about? I've found only the bug report 1242.

What spi stays for in "com.jogamp.opengl.util.texture.spi"?

On 12/15/2015 09:03 AM, elect [via jogamp] wrote:
> What spi stays for in "com.jogamp.opengl.util.texture.spi"?

Service Provider Interface

~Sven

This is the best I could do given the circumstances

https://github.com/elect86/joglTexture/blob/master/joglTexture/src/texture/spi/dx/DdsImage.java#L744-L756

I will have to do the same for all the other line. And this is just for getting the right format..

I will have to write a whole new function for every possible translation I will have to perform..

I also modified the math package.. I am thinking to invest some energy also in the creation of a robust math library.. would it be possible to have Vec and Mat classes or is it out of discussion because of the class allocation?

I'd like to get some feedbacks about both however :)

Hi

Please document a bit more, the first test using "IntUtil.equal" seems to be out of nowhere, there should be a comment explaining it.

There are already com.jogamp.opengl.math.Matrix4 and com.jogamp.opengl.math.VectorUtil.