Quantcast

Process database table (or complex data)?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Process database table (or complex data)?

bojan
Hi!

I am newbie to jocl. I was given a task to do some data processing on GPU, so jocl sounded like a good solution. What I need to do is to load a table from database (actually a few of them), do some processing and return a results (usually one, filtered, table).
In Java I load database table to List<List<Object>>   ("List" is one row from database, it contains different data types (numbers, dates, strings,...) ).

So my question if how to pass these input data to jocl script? Currently I didn't find a way to pass it as it is, so I am loading column by column, fill buffer with it and giving it to a script. It works ok for numbers, but I can do the same thing with string or dates (at least not yet). And to be honest - I don't like this approach - if I have 3 tables, each of them has 5 columns - it means 15 arguments in my script - not so nice. Any suggestion how to do this?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Process database table (or complex data)?

gouessej
Administrator
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Process database table (or complex data)?

bojan
That method takes CLBuffer<? extends java.nio.Buffer>  as argument. In that way I can pass just some "simple" data - floats, ints, but how to pass strings, dates, or even one (or multiple) row from db (containing list of elements of different types)? Maybe extend java.nio.Buffer?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Process database table (or complex data)?

gouessej
Administrator
You should only pass the data that you're going to use in the computation, I don't think that you need to pass dates, strings, ...

You should extract the simple data from each row to pass them by using several buffers, one buffer per field, one value per row. Let's assume that you have three rows and two fields, you'll build two buffers and you'll put three values in each buffer.

It's possible to use "structs" in OpenCL but maybe it's overkill in your case.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Process database table (or complex data)?

bojan
Well, I need to use string and dates also. To make things short - I have to do something like sql join, but programmatically (in memory). So I need to load all data from all tables, and do something like
select t1.xxxx, t2.xxx, t3.xxx from table1 t1, table2 t2, table3 t3 where tj.date1 > t2.date2 and t2.str1 = t3.str2 and .....

We are doing this in pure java - it works, but sometimes we have performance issues, our idea was to try to run it on gpu and see how it performs, but now I am afraid is jocl capable of taking all data that we have (or use) and processing them...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Process database table (or complex data)?

gouessej
Administrator
These are the data types supported by OpenCL:
https://www.khronos.org/registry/cl/sdk/2.1/docs/man/xhtml/dataTypes.html

Don't blame JOCL. You can decompose your data into simple data to make it work instead of insisting on passing dates and strings. You can convert a date into a timestamp.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Process database table (or complex data)?

Wade Walker
Administrator
In reply to this post by bojan
Hi Bojan,

I've seen many articles and presentations on acceleration of database operations using OpenCL (and CUDA, and GPUs in general); see for example http://developer.amd.com/wordpress/media/2013/06/1738_final.pdf, so I know it's possible. It does look like there's a bit of difficulty involved, due to the nature of GPUs :) Bu since JOCL just maps OpenCL into Java, anything that works in OpenCL should also work in JOCL (if not, just let me know, since I'm the maintainer).
Loading...