Graph Type Rendering Updates (Graph UI)

Added some details in this blog https://jausoft.com/blog/2023/02/12/graph-type-rendering-update-1/

This is a work in progress.

... more to come

Update 2, see https://jausoft.com/blog/2023/02/13/graph-type-rendering-update-2/

Besides handling all Type Rendered OutlineShape in independent font em-size units, the sweet ole UI demos have been updated accordingly.

How do you plan to handle CSS descriptions?

Update 3, see https://jausoft.com/blog/2023/02/16/graph-type-rendering-update-3/

.. one bug has been fixed and a little demo-tool to investigate the remaining.

Parsing the text and producing OutlineShapes for example, but I have not really planned this yet as .. tada .. no funding.

Just in case, if there is no slim-parser available, I will just write my own.
Idea is to use CSS object references for shapes at least.

gouessej wrote

How do you plan to handle CSS descriptions?

Maybe JLessC and LessCSS4j are worth a try, they can be used with plain CSS.

Update 4, see https://jausoft.com/blog/2023/02/17/graph-type-rendering-update-4/

Update four in the Graph Type Rendering Saga. Here just a quick update on fixing serif font glyphs like g and æ and inner shape or holes within OutlineShape in general.

gouessej wrote

Maybe JLessC and LessCSS4j are worth a try, they can be used with plain CSS.

Thank you, looks interesting. I may tend to a runtime parser though .. but we will see.
More fixing to do still :)

Graph Curve Renderer Overflow

One remaining Graph Curve Renderer issue is having 'too much text',
overflowing as in looping from the end text-block to the beginning with garbage.

This occurs since our ES2 VBO GLRegion implementation uses UNSIGNED_SHORT to store vertex-indices
and we hit the roof with a simple ~700 character text.

- Using UNSIGNED_INT pushes out this limitation for 'a good while' :)

- Since ES3, glDrawElements(..) does support UNSIGNED_INT, good.

- Perhaps need to find an API way to tell the user ...

- Perhaps partitioning the region into more than one?

- Perhaps reduced index count by properly reusing shapes,
  same way as we use it before triangulation for font glyphs etc?

- There is room for improvements and perf enhancements in this area ..

+++

So while the above resolves the overflow issue, there are still glyph rendering bugs
with some fonts ..

Here a little snapshot showing the overflow being fixed (otherwise FreeMono text would have been broken),
but also shows some issues left .. (also w/ FreeSerif ..)

Little list of other bugs:
- Some glyphs still buggy (FreeMono, FreeSans, but FreeSerif is OK)
  - perhaps still some winding issues?
- Layout broken for FreeSerif, ' ' <space> spacing is tooo wide.

OK, screenshot now from here (not killing the text flow here)
https://jausoft.com/blog/wp-content/uploads/2023/02/text-vbaa1-msaa0-FreeMono_Regular-text1-S04-Z0000-snap06-1628x640-1.png

More Updates .. https://jausoft.com/blog/2023/02/19/more-graph-type-rendering-updates/

Performance analysis and changes, done w/ 1st iteration.

Graph Performance: 1296 Glyphs processed in 10.25ms on Raspi 4b

Graph Perf Update: 1296 chars to Region per Frame:
-  RaspiPi4 10.25ms (regioned) + 3.4ms (draw)
-  PC 2.97ms (regioned) + 0.36ms (draw)

Performance update from commit 607eb99b9cad227dd7be6d149c6b6cf57d060c35
(Note: There I mentioned the total duration for 20 frames, not per frame)

regioned is the process where all single pre-computed OutlineShape instances per Glyph are processed to become one Region. This process includes our Font layouting and Region.addOutlineShape().
Region.addOutlineShape() itself performs the triangulation of the shapes, compounding of all vertices and pushing all data down to the VBO buffer, ready to be rendered. Hence, the crucial Graph hotspot.

... more in the blog

Edit: I have updated the logs and numbers, now with vsync swap-buffers turned off. However, this seems to make no difference on the Raspi 4b. Also added screenshots of the visual performance analysis.

Why does VisualVM still show swapBuffers as a performance hog even with vsync disabled?

good question and I did ask this myself :)
(GL sync, actual copy of bytes w/ dma or not instead of page-flip, profiler sampling-bias...???)

Our internal counter can show where its comping from and will refine it a bit,
the 'sample bias' of both tools is surely a PIA.

Further tests showed that I had FSAA enabled. So after disabling perf went up a little ..
this could have been one ..

Goal was to at least see that non of our curve processing are ruining the show
and this has been achieved.

OK, I promoted GraphUI to JOGL ... its a WIP of course,
but let's make it more available to play with it ..

commit 3131eaaf5272ca3f0011e334eb08c6ba68702a6c

    GraphUI: Promote API to JOGL via graphui.jar or within any jogl-all*.jar (WIP)
   
    Root package is 'com.jogamp.graph.ui.gl', i.e. a sub-package of Graph denoting UI and OpenGL usage.
   
    Implementation will stay small, hence relative files size costs are minimal.
   
    Source and build is in parallel to nativewindow, jogl and newt
    and has a dependency to all of them.
    The NEWT dependencies are merely the input listener ..

Just in case anybody wonders .. current tasks for me re GraphUI are
- Simplify API, enable a more hidden way for resource details less burden, but keep it open
- Make 'shape' interaction more reactive: resize, drag, ..
- Demo
- Have a few more 'shapes' maybe
- Show animated text like
  - Push movie subtitles to screen
  - Modern text animation w/ moving, acceleration, collision, sheering, ...
- Start the 'retention' mode reading description from a file

Little update from the top of my head ..

Graph and GraphUI API has been simplified, see UISceneDemo00 and UISceneDemo01.

Shape move and resize fixed, would need abstraction of alignment for better results on multi-line text etc.

Overall lifecycle tackled, prepared for (text) animation using Label.

- Shape GLButton lifecycle fixed.
- Shape MediaButton visible at any state of GLMediaPlayer (text image pre stream arrival)

.. I would say, it is usable now.

Will show video and blog updates soon.

I have two questions for you:
- Is there another API doing something vaguely similar that could be used as an example in terms of performance and simplicity or a standard?
- Would it be difficult to make Graph UI work with Vulkan or WebGPU?

Keep up the good work.

gouessej wrote

I have two questions for you:
- Is there another API doing something vaguely similar that could be used as an example in terms of performance and simplicity or a standard?

I use a simple non-graph approach so far, but w/ z-ordering in display() of Scene,
no occlusion testing.

Back then, I made a similar approach for a non-geom scene graph,
and spatial occlusion/clip tests.
All this more based on core GL 'self made' features and attributes.
Mumble ..

This GraphUI API is KISS, nothing really special but its implementation w/ Graph.

I will add the free shape form thingy via a format (instead of just a round button.
Goal is to use these shapes and text for a flexible UI.

Yes, the typical UI elements checkbox, drop-down bla .. shall also come.

- Would it be difficult to make Graph UI work with Vulkan or WebGPU?

Keep up the good work.

As mentioned, only using core GL, so it should be technically easy.
Just shader and data settings, all setup and processing magic is universal.

I am offering porting this to like C++ or even Vulkan w/ a JOVK (JOGL Vulkan lol)
if anybody likes ... you all know how to contact me for nice contracts :)

First demo video update GraphUI Enhanced Scene and Shape Interactions & MediaButton

Today I will add a few blog posts regarding updates in the Graph Type Rendering and UI Saga. This first one covers enhancements with user interaction. Earlier we used a scroll-wheel to translate the Shape in the Scene on its Z-axis. Now we can use a build-in 1-pointer drag-zoom, actually scaling the Shape.

Additionally, GLMediaPlayer has been enhanced to deliver a test-screen (texture image) until the stream delivers the first video packets. This allows using it at any state and removes earlier complications to code its status, i.e. whether the stream is available or not. This is used within GraphUI’s MediaButton, see video below.

... will follow up with text/type animation updates today.