Hello All . I was wondering if anyone has had any experience with an abstraction or some built in process being able to handle character recognition on any platform. I mostly interested in running a test on Linux Ubuntu. What I would like to do is have PD react to marks or characters being picked up by a webcam. I suppose the web-cam would have to capture an image (pixvideo, piximage) and pass that to an ocr engine and then read a text string from the application to be processed by PD in one way or another.
Any help of pointers would be greatly appreciated.
Alvin
Hello,
You can do this with the use of artificial neural network (for character recognition). There are externals for Pd : http://pure-data.svn.sourceforge.net/viewvc/pure-data/trunk/externals/ann/ ++
Jack
Le mercredi 01 juin 2011 à 17:09 +0200, Alvin Google a écrit :
Hello All . I was wondering if anyone has had any experience with an abstraction or some built in process being able to handle character recognition on any platform. I mostly interested in running a test on Linux Ubuntu. What I would like to do is have PD react to marks or characters being picked up by a webcam. I suppose the web-cam would have to capture an image (pixvideo, piximage) and pass that to an ocr engine and then read a text string from the application to be processed by PD in one way or another.
Any help of pointers would be greatly appreciated.
Alvin
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On Wed, 1 Jun 2011, Jack wrote:
You can do this with the use of artificial neural network (for character recognition). There are externals for Pd : http://pure-data.svn.sourceforge.net/viewvc/pure-data/trunk/externals/ann/
Is it just me, or it sounds like it's going to take a lot of preprocessing before you can even think of feeding it to a neural network ?
Human vision is made of a lot more layers of neurons than we can hope to deal with in artificial networks.
At what angles should characters be recognised ?
Which colour on which colour ?
You better settle those things first, so that you can figure out how you can reduce your data beforehand.
But making an OCR using ANN is a lot lot more work than using an OCR library. Making a Pd-to-OCR-library interface is less work than making an OCR abstraction library... and it isn't necessarily because Pd would be bad at it (I don't know about that). It's more because it takes a lot of knowledge to make an OCR library from nearly scratch.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Super, I will check out the link and sorry for the double post.
It is in the early concept stages, but I was thinking that the user interface would be a piece of glass/plastic the input device would be a webcam on the inside of a box/resonator. The ambient light outside of the box (no pun intended) would be enough of a difference to read the opaque markings as parse-able characters. The user would draw on an opaque surface and puredata would be able to make something/interpret this input and respond in some way. It may be that I have to simplify this to gestures which I think puredata is also capable of doing. Reading a webcam capture of the entire active area and the comparing that to a prior state.
Colour would open up other modes, but at this stage binary is what I am looking for, just black and white.
I would be happy with the simplest method and the easiest to compute.
Thanks Again.
On 1 June 2011 17:49, Mathieu Bouchard matju@artengine.ca wrote:
On Wed, 1 Jun 2011, Jack wrote:
You can do this with the use of artificial neural network (for character
recognition). There are externals for Pd : http://pure-data.svn.sourceforge.net/viewvc/pure-data/trunk/externals/ann/
Is it just me, or it sounds like it's going to take a lot of preprocessing before you can even think of feeding it to a neural network ?
Human vision is made of a lot more layers of neurons than we can hope to deal with in artificial networks.
At what angles should characters be recognised ?
Which colour on which colour ?
You better settle those things first, so that you can figure out how you can reduce your data beforehand.
But making an OCR using ANN is a lot lot more work than using an OCR library. Making a Pd-to-OCR-library interface is less work than making an OCR abstraction library... and it isn't necessarily because Pd would be bad at it (I don't know about that). It's more because it takes a lot of knowledge to make an OCR library from nearly scratch.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
On Wed, 1 Jun 2011, Alvin O'Sullivan wrote:
It is in the early concept stages, but I was thinking that the user interface would be a piece of glass/plastic the input device would be a webcam on the inside of a box/resonator. The ambient light outside of the box (no pun intended) would be enough of a difference to read the opaque markings as parse-able characters.
Ok, this is two-dimensional reading, which is already a lot more ordinary. If characters are upright (non-rotated) it may be even more ordinary. This means it'll be easier to find existing support for reading characters, or to (re)invent that support.
Beyond that, I don't know much more about OCR.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
On 6/1/11 8:54 PM, Mathieu Bouchard wrote:
On Wed, 1 Jun 2011, Alvin O'Sullivan wrote:
It is in the early concept stages, but I was thinking that the user interface would be a piece of glass/plastic the input device would be a webcam on the inside of a box/resonator. The ambient light outside of the box (no pun intended) would be enough of a difference to read the opaque markings as parse-able characters.
Ok, this is two-dimensional reading, which is already a lot more ordinary. If characters are upright (non-rotated) it may be even more ordinary. This means it'll be easier to find existing support for reading characters, or to (re)invent that support.
Beyond that, I don't know much more about OCR.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
I've not used the neural net libraries/abstracts before so I will give it a try and see how it goes. It sounds as though having a good grid size to input method resolution would be the first step. Some of the image processing can be handled by the web-camera driver: contrast, flipping the image etc. The interface will be part of a sculpture that act as resonator, so depending on the input I can catch and process my patch will respond with auditory feedback. The key is to keep the user interested and willing to play with this thing.
Marks or gestures would be enough if character processing were not practical.
I will also have to some how recorded the different session states later to be recalled when a user is not available.
Thanks again for all of your help.
A
Is it just me, or it sounds like it's going to take a lot of preprocessing before you can even think of feeding it to a neural network ?
Black/white thresholding and resolution reduction, that's it.
Human vision is made of a lot more layers of neurons than we can hope to deal with in artificial networks.
It sure is. Luckily you only need a few to do some basic OCR. You could try a grid of something like 10x10 pixels just to start, which would require 100 input neurons. The lower the resolution, the higher your error rate will be, but you can find good compromises.
At what angles should characters be recognised ?
Well, that's the beautiful thing about neural nets—it just depends on how you train the net. If you want the net to be able to recognize tilted letters, you can add tilted letters to the training sets. It can affect
But making an OCR using ANN is a lot lot more work than using an OCR library. Making a Pd-to-OCR-library interface is less work than making an OCR abstraction library
Agreed, strongly. OCR is a really excellent exercise in neural nets (look up Self-Organizing Maps or Kohonen networks), but it's a lot of work. It would be faster by far to set up an interface as Mathieu suggests. However, if you want to go through with it anyway, I'd love to help!
It looks like the defacto open ocr lib is Tesseract: http://code.google.com/p/tesseract-ocr/ This would be great to have in Pd.
Incidentally, I can't see any reason why Pd would be "bad at it," since the ANN external uses the C FANN library, which is likely what any other library would use in the first place.
±tedb0t
On Jun 1, 2011, at 11:49 AM, Mathieu Bouchard wrote:
On Wed, 1 Jun 2011, Jack wrote:
You can do this with the use of artificial neural network (for character recognition). There are externals for Pd : http://pure-data.svn.sourceforge.net/viewvc/pure-data/trunk/externals/ann/
Is it just me, or it sounds like it's going to take a lot of preprocessing before you can even think of feeding it to a neural network ?
Human vision is made of a lot more layers of neurons than we can hope to deal with in artificial networks.
At what angles should characters be recognised ?
Which colour on which colour ?
You better settle those things first, so that you can figure out how you can reduce your data beforehand.
But making an OCR using ANN is a lot lot more work than using an OCR library. Making a Pd-to-OCR-library interface is less work than making an OCR abstraction library... and it isn't necessarily because Pd would be bad at it (I don't know about that). It's more because it takes a lot of knowledge to make an OCR library from nearly scratch.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC _______________________________________________ Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Whoops, didn't finish this sentence, I was going to delete it anyway, please disregard :)
Well, that's the beautiful thing about neural nets—it just depends on how you train the net. If you want the net to be able to recognize tilted letters, you can add tilted letters to the training sets. It can affect
±±t3db0t
Le mercredi 01 juin 2011 à 12:51 -0400, Tedb0t a écrit :
Is it just me, or it sounds like it's going to take a lot of preprocessing before you can even think of feeding it to a neural network ?
Black/white thresholding and resolution reduction, that's it.
I also use grey images and it was working fine.
Human vision is made of a lot more layers of neurons than we can hope to deal with in artificial networks.
It sure is. Luckily you only need a few to do some basic OCR. You could try a grid of something like 10x10 pixels just to start, which would require 100 input neurons. The lower the resolution, the higher your error rate will be, but you can find good compromises.
I was working with a grid of 16x16 pixels and it was OK for me.
At what angles should characters be recognised ?
Well, that's the beautiful thing about neural nets—it just depends on how you train the net. If you want the net to be able to recognize tilted letters, you can add tilted letters to the training sets. It can affect
Yep, if you train your NN with angles, deformations, etc. You should get good results. ++
Jack
But making an OCR using ANN is a lot lot more work than using an OCR library. Making a Pd-to-OCR-library interface is less work than making an OCR abstraction library
Agreed, strongly. OCR is a really excellent exercise in neural nets (look up Self-Organizing Maps or Kohonen networks), but it's a lot of work. It would be faster by far to set up an interface as Mathieu suggests. However, if you want to go through with it anyway, I'd love to help!
It looks like the defacto open ocr lib is Tesseract: http://code.google.com/p/tesseract-ocr/ This would be great to have in Pd.
Incidentally, I can't see any reason why Pd would be "bad at it," since the ANN external uses the C FANN library, which is likely what any other library would use in the first place.
±tedb0t
On Jun 1, 2011, at 11:49 AM, Mathieu Bouchard wrote:
On Wed, 1 Jun 2011, Jack wrote:
You can do this with the use of artificial neural network (for character recognition). There are externals for Pd : http://pure-data.svn.sourceforge.net/viewvc/pure-data/trunk/externals/ann/
Is it just me, or it sounds like it's going to take a lot of preprocessing before you can even think of feeding it to a neural network ?
Human vision is made of a lot more layers of neurons than we can hope to deal with in artificial networks.
At what angles should characters be recognised ?
Which colour on which colour ?
You better settle those things first, so that you can figure out how you can reduce your data beforehand.
But making an OCR using ANN is a lot lot more work than using an OCR library. Making a Pd-to-OCR-library interface is less work than making an OCR abstraction library... and it isn't necessarily because Pd would be bad at it (I don't know about that). It's more because it takes a lot of knowledge to make an OCR library from nearly scratch.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC _______________________________________________ Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On Wed, 1 Jun 2011, Tedb0t wrote:
Well, that's the beautiful thing about neural nets—it just depends on how you train the net. If you want the net to be able to recognize tilted letters, you can add tilted letters to the training sets.
That would take a damn lot more elements in the training set !
It looks like the defacto open ocr lib is Tesseract: http://code.google.com/p/tesseract-ocr/%C2%A0%C2%A0This would be great to have in Pd.
What a name. I mean, it's geometry-related, but Tesseracts are about four-dimensional spaces. :}
Incidentally, I can't see any reason why Pd would be "bad at it," since the ANN external uses the C FANN library, which is likely what any other library would use in the first place.
Reasons why a language would be bad at it are obviously because of what would be different, and not what would be the same. FANN would be the same, and the rest would be different.
BTW, [ann_som] is not using FANN, did you know that ?
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC