B4J Question How to draw zoom-out figure of data samples but keep the key points shown?

[kimstudio]

Not a directly B4J quesiton but could be interesting:

I have 8000 data samples and want to draw a whole plot figure, but my canvas width is only 200 pixels, I can draw one sample every 40 samples for one pixel. The problem is almost all data sample values are zero, except that values of sample index 46, 846, 1646, etc. are 100.0, then my plot will not show these key points. Using Python or Matlab to plot the key points will always be drawn no matter how small the figure is. What is the logic/algorithm behind to fulfill this function?

@klaus @William Lancee maybe you are interested...

Also I got nothing from google, the problem is also how to phrase this question, since it is kind of hard to explain...

 

[teddybear]

Perhaps you can use the sample mean per 40 data samples to draw a whole plot figure

 

[klaus]

You can do it with the xChartMini library.

The code:

Private Sub Draw
    Private i As Int
    
    xChartMini1.AddLine("Test", xui.Color_Blue)
    
    For i = 0 To 8000
        If (i - 46) Mod 800 = 0 Then
            xChartMini1.AddLinePointData(i, 100, i Mod 1000 = 0)
        Else
            xChartMini1.AddLinePointData(i, 0, i Mod 1000 = 0)
        End If
    Next
    xChartMini1.DrawChart
End Sub

You could also do it on your own with a canvas.
Draw the scale and:
Either draw all 8000 data points with a line in between.
Or draw a horizontal line at Y = 0 and draw a vertical line when y = 100.

 

[Daestrum]

You could use a viewport - possibly.

Example attached - it's rough but gives the general idea. also the canvas need not be on the mainform, in the example it is just to check the lines looked the same.

 

[kimstudio]

The lesson is I have an old style programming mind that pixel coordinate should be an integer, actually in javafx the coordinate can be double in Drawline. So if I draw 8000 samples in a 200 width figure I can still use double coordinate scaled from 8000 to 200 and draw all 8000 samples, then every points will be drawn on the figure. Thanks @klaus for your code in the xChartMini library.

The problem is draw all 8000 samples in 200 width is time consuming, beacuse same coordinate will draw 40 times, that's a reason to first downsample 8000 samples to 200 and try only draw 200 samples, but with cost of losing these key points. I was thinking there are some algorithms behind that to find these key points during downsample then explicitly draw these points. I think one way to accelerate drawing all samples is that if Drawline is the bottleneck the we can compare if same coordinates will draw same values then only draw once.

 

[kimstudio]

Perhaps you can use the sample mean per 40 data samples to draw a whole plot figure

I tried this method but the 100 will be averaged with zeros so the value is not correct. I also considered to find the max/min and difference with adjacent points to figure out whether it is a key point or not, but I think it is not correct answer. Check klaus' code.

 

[klaus]

The problem is draw all 8000 samples in 200 width is time consuming, beacuse same coordinate will draw 40 times, that's a reason to first downsample 8000 samples to 200 and try only draw 200 samples,

What is less time consuming, draw the 8000 points or downsize the points and draw them.
As already suggested, you could do it.
Draw the scale, draw the line at zero go through the 8000 points, check if the value is bigger than zero and draw the line.
Does this take less time, I do not know. Is it disturbing for the user, I do not know.
I am astonished that you want to show a chart with this specific data.
The advantage of xChartMini is, i did not suggest you the xChart library which is much bigger, that it remains universal.
If you draw a specific chart like explained above, it is specific to your application, yes, with less code, but it needs more time to create it
Then, what is the best solution, you need to analyse your needs on speed perception by the user and time to create it.
It is not complicated, but must be done.

 

[kimstudio]

This dataset I created is just an example to show the problem. Indeed we can analyze the requirement to select plot method. For example, to draw a real-time recording audio wave signal 8000 samples on a 200 pixel width canvas, I know some key points are not that important for user then I can choose only draw 1 sample every 40 samples so for each update I only draw 200 samples to save drawing time. However if I don't want to miss any key points like the data I created or a ECG waveform then I will draw all 8000 samples even on a 200 width canvas.

 

[William Lancee]

A bit late to this discussion.

1. Basically you have 200 bins.

2. You have to characterize the 40 data points for each bin.

3. Most of the points in each bin are zero.

Is that correct?

If so, how to characterize a highly skewed distribution in each bin?

The average won't work. There are many solutions in the literature.

One possibility: add 1 and take the log, and display the average log for each bin.

 

[William Lancee]

About EEG.

I assume you are using a threshold for a stimulus response profile. Usually they are averaged over each stimulus, so the zero values are not relevant.

If that is the case you need to provide more info about the problem.

 

[klaus]

As William Lancee suggests, it would be interesting to know the kind of signals you want to display.and look for the best solution.
Could you provide a sample?

 

[kimstudio]

The signal is from BCI invasive microelectrode. It's usually sampled at 30Khz and there are some 1-2ms spikes on signals like the plot figure below.

I want to draw the raw signal in real-time and don't miss the spikes even on a small width canvas. Now I got the answer which is just draw every sample and don't do downsampling before drawing. Thanks all.

 

[William Lancee]

Surprisingly, I also saw this type of data from a single neuron signal in the early 70's.
Glad you found a solution.

I worked in a lab doing microiontophoresis.

You may want to eliminate the slow wave with a filter.