B4J Question Recognize playing cards using opencv

jroriz · Oct 19, 2022

Could someone help me to write a program to recognize playing cards using opencv?
The cards always appear in the same place and always with the same pattern.
What can change is the window size, which the user can freely change, but always in the same proportion.
Where the cards are is not my problem. I can with basic math know the size of the window and where
the cards would be.

Then I would crop the part of the screen where the cards are and apply opencv to the cropped image.

emexes · Oct 20, 2022

Why not roll your own?

1/ Scan the image looking for white-ish pixels (ie playing card background)
2/ Scan down from those pixels and count how many contiguous white-ish you have, to get an idea of the size of the card
3/ For plausible run lengths (eg the large cards in your example are 62 pixels tall, and the small cards are 50 pixels tall, so let's say anything in the range 40-to-90 pixels)
3a/ Scan left from the vertical middle of the run, until you hit a non-white-ish pixel, so that we now know where the top, bottom and left of the card are
4/ Probe a grid of pixels across the expected card area for white, red or black pixels, and compare them to what you'd expect for each of the 52 possible cards (plus jokers?)

bonus 1: it looks like you don't have to scan the whole image, because cards only appear in eg 7 areas above (6 players plus 1 dealer) and we don't need to check the corners, or the areas to the left of the players at the top and bottom

bonus 2: once you've located a card, you don't need to scan the rest of that area of the card

thought: the scan for "white-ish" pixels might be better done for "white-to-reddish or white-to-blackish" pixels ie if you have green or blue or yellow or any other hue that isn't red or black or white, then pixel isn't part of a card

thought: best to identify using left-third of card only, thanks to the sometimes overlaying by other cards

edit: hang on, you can only see your own cards and the dealer's cards, right? Not the other players? So now there are only two areas we need to scan. Bonus!!!

emexes · Oct 20, 2022

Righto...

The pixels of the black cards are RRGGBB where the byte values RR GG and BB are identical eg 0C0C0C or 3A3A3A but let's allow for a bit of spread eg +/- 3 between the three values.

The pixels of the red cards have RR values of E3..FF (so let's say DB..FF), and GG and BB values that are within 6 of each other and in range of 02..F2, so let's say within 8 of each other and 00..F8.

Magma · Oct 20, 2022

...hmmm.. You can use tesseract-ocr cmd line (before some years used it with very big success)... ofcourse will be better to grab card by card ... if they have fixed spaces...

Tesseract documentation

Documentation

tesseract-ocr.github.io

edit: as I see in forum... there are members tried and create class/libs... search the forum - may be all these can help you

emexes · Oct 20, 2022

first baby steps... looking for black cards, using she'll-be-right-mate parameters randomly picked out of thin air...

clearly need to fine-tune with the ace

emexes · Oct 20, 2022

one step forward, two steps back... but hey, that ace is looking better ?

TILogistic · Oct 20, 2022

emexes said:
Why not roll your own?

1/ Scan the image looking for white-ish pixels (ie playing card background)
2/ Scan down from those pixels and count how many contiguous white-ish you have, to get an idea of the size of the card
3/ For plausible run lengths (eg the large cards in your example are 62 pixels tall, and the small cards are 50 pixels tall, so let's say anything in the range 40-to-90 pixels)
3a/ Scan left from the vertical middle of the run, until you hit a non-white-ish pixel, so that we now know where the top, bottom and left of the card are
4/ Probe a grid of pixels across the expected card area for white, red or black pixels, and compare them to what you'd expect for each of the 52 possible cards (plus jokers?)

bonus 1: it looks like you don't have to scan the whole image, because cards only appear in eg 7 areas above (6 players plus 1 dealer) and we don't need to check the corners, or the areas to the left of the players at the top and bottom

bonus 2: once you've located a card, you don't need to scan the rest of that area of the card

thought: the scan for "white-ish" pixels might be better done for "white-to-reddish or white-to-blackish" pixels ie if you have green or blue or yellow or any other hue that isn't red or black or white, then pixel isn't part of a card

thought: best to identify using left-third of card only, thanks to the sometimes overlaying by other cards

edit: hang on, you can only see your own cards and the dealer's cards, right? Not the other players? So now there are only two areas we need to scan. Bonus!!!

The method you describe.

Is it SIFT (Scale Invariant Feature Transformation)?

Magma · Oct 20, 2022

1) I must say that i don't know poker or any game it is... but i like the "concept"

2) You can grab only this region:

3) Give a try to tesseract-ocr to read/get it like this: "8 9 10 6 7"... so with regex.split you will split at an array int...

4) You can getpixel-color with specific distance from this image to take the color of cards... "or" check/recognize for those symbols too: ---> so you will get an array with those too...

TILogistic · Oct 20, 2022

?

OpenCV: Introduction to SIFT (Scale-Invariant Feature Transform)

docs.opencv.org

TILogistic · Oct 20, 2022

Questions.

uses images to display the cards?

If so, you already have the patterns to look for.

emexes · Oct 20, 2022

TILogistic said:
The method you describe.

Is it SIFT (Scale Invariant Feature Transformation)?

Well, I wouldn't know about that fancy-named stuff, but certainly I was headed towards handling cards of different sizes, within a specified size range:

emexes said:
3/ For plausible run lengths (eg the large cards in your example are 62 pixels tall, and the small cards are 50 pixels tall, so let's say anything in the range 40-to-90 pixels)
3a/ Scan left from the vertical middle of the run, until you hit a non-white-ish pixel, so that we now know where the top, bottom and left of the card are

and then once we'd determined the size of the card from the rectangle of white-ish card background, then the plan was to scale the grid accordingly:

emexes said:
4/ Probe a grid of pixels across the expected card area for white, red or black pixels, and compare them to what you'd expect for each of the 52 possible cards (plus jokers?)

One thing to note is that, if you look closely, the rendering of the dealer's three clubs is not 100% pixel-for-pixel the same for each, which I assume is due to being positioned on non-integer pixel coordinates. No problem, as long as you spot it before tripping over it. ?

I'm happy that it is plausible to do the Optical Card Recognition "manually", so I've moved back to what I'm supposed to be doing, which coincidentally puts food on the table too. If @jroriz somehow gets super-bogged down, then I'll be back after the weekend if you need help towing him out. ?

jroriz · Oct 30, 2022

Thanks everyone for the suggestions.
I've been really busy the last few days.

OCR is what I'm currently using, however when the screen size gets smaller it's a nightmare.

So OCR would be enough from a certain resolution. And for other poker apps or even app updates generate errors in detection via OCR.

Detecting suits is not my problem. My problem is to determine the value of the cards.

So I think my best bet would be to crop part of the image and count the pixels that tend to be white in color.

But something tells me that if I could teach opencv to recognize the cards it would be easier, since the cards are always the same.

jroriz · Oct 30, 2022

emexes said:
one step forward, two steps back... but hey, that ace is looking better ?

View attachment 135039

How did you get this results?

And how can I teach the application the cards value?

jroriz · Oct 30, 2022

Magma said:
1) I must say that i don't know poker or any game it is... but i like the "concept"

2) You can grab only this region:
View attachment 135041

3) Give a try to tesseract-ocr to read/get it like this: "8 9 10 6 7"... so with regex.split you will split at an array int...

4) You can getpixel-color with specific distance from this image to take the color of cards... "or" check/recognize for those symbols too: ---> so you will get an array with those too...
View attachment 135042

I've tried things like that and the results are bad.
Even when I "clean" the image, leaving only the value of the cards, the result is very erratic.
My best result was reading letter by letter, even so there are many errors.

Magma · Oct 30, 2022

jroriz said:
View attachment 135424
I've tried things like that and the results are bad.
Even when I "clean" the image, leaving only the value of the cards, the result is very erratic.
My best result was reading letter by letter, even so there are many errors.

Hi there... I don't remember if it is possible to train it with default pictures...

...search a little the web

Magma · Oct 30, 2022

My last try with tesseract was before 6 years... reading old phone catalogs..with 85% success

emexes · Oct 31, 2022

jroriz said:
OCR is what I'm currently using, however when the screen size gets smaller it's a nightmare.

Yeah, the pixel smudging that starts happening at those low character sizes ain't helping.

jroriz said:
How did you get this results?

Magic. ?

jroriz said:
And how can I teach the application the cards value?

If you grab me screen captures of all the cards (13 values x 4 suits + jokers?) at the sizes you're working with, I'll give it a go. Might be weekend before I get to it, though.

Pixel-for-pixel screen capture as PNG, don't have to be of single cards (ie, overlaid cards like you see in your game is fine), don't have to be neatly aligned, don't have to be in order, happy to have multiple captures of same value+suit card so can see if there is there is smudging due to sub-pixel positioning, eg a one-pixel black line might be 5 pixels white, white, black, white, white, or if shifted right by one-third of a pixel: white, white, dark grey, light grey, white.

For cards that can be overlaid, it'd be good to have captures of them all whilst overlaid (plus spurious duplicates of the not-overlaid top card, but I'd probably ignore those, other than making sure that the recognition of overlaid cards by their left side, works for non-overlaid situations too).

Magma · Oct 31, 2022

jroriz said:
View attachment 135424
I've tried things like that and the results are bad.
Even when I "clean" the image, leaving only the value of the cards, the result is very erratic.
My best result was reading letter by letter, even so there are many errors.

...hmm do you get always the same results for the cards from tesseract..

For example if 19 is 9.. always replace the string with 9...

If [0 is 10... replace with 10

kimstudio · Oct 31, 2022

I suggest to just use template matching function from opencv to realize this: cv.matchTemplate()

Prepare template of every card and match each template with the image captured, better to cut the area with cards in only as target image as mentioned this is not a problem, to save computation time. The size of template must be equal to the card size in target image, if not then resize the target image to make the card size matching the template size. The max of all matching values indicates the recognized card.

This would be the template to prepare for every card:

The template match is scale variant so the size of number 8 must be equal to the size in the target image, if not then resize according to screen size or portion as mentioned.

jroriz · Oct 31, 2022

Magma said:
...hmm do you get always the same results for the cards from tesseract..

For example if 19 is 9.. always replace the string with 9...

If [0 is 10... replace with 10

That's what I did.

B4J Question Recognize playing cards using opencv

Active Member

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Expert

Active Member

Active Member

Active Member

Expert

Expert

Expert

Expert

Active Member

Active Member

Similar Threads

Privacy & Transparency

Privacy & Transparency