Dear List,
I did a little bit of machine learning with neural network when I was in school, and I'd like to try it on sounds. What I'd like to do is to identify patterns, types of sounds, like "people talking", "loud, compressed rock music", etc.
Is that feasible ? I found this library on the web : https://github.com/Cycling74/ml-lib But I have no clue how to use it.
Do you have any suggestions on where to start ? Can I feed it sound files ? Or do I need to extract some "indicators" from it (loudness, spectrum, or something) ?
Thanks in advance for your help !
Cheers,
Pierre.
On 21/07/2016 12:08, Pierre Massat wrote:
Dear List,
I did a little bit of machine learning with neural network when I was in school, and I'd like to try it on sounds. What I'd like to do is to identify patterns, types of sounds, like "people talking", "loud, compressed rock music", etc.
If I understand correctly, maybe the keyword you're after is "automatic music classification"? (to which you could add e.g. "machine learning" "pure data" etc.). In this case there is loads of stuff... A good starting point (other than google) could be: http://www.ismir.net/society.html
Hope this helps. Lorenzo.
Is that feasible ? I found this library on the web : https://github.com/Cycling74/ml-lib But I have no clue how to use it.
Do you have any suggestions on where to start ? Can I feed it sound files ? Or do I need to extract some "indicators" from it (loudness, spectrum, or something) ?
Thanks in advance for your help !
Cheers,
Pierre.
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Please note that most applications of neural nets are non-realtime, e.g. not in the same domain as Pure Data. The evaluation of neural networks can be, but the training never is. best, Thomas
Am 21.07.2016 um 14:37 schrieb Lorenzo Sutton lorenzofsutton@gmail.com:
On 21/07/2016 12:08, Pierre Massat wrote:
Dear List,
I did a little bit of machine learning with neural network when I was in school, and I'd like to try it on sounds. What I'd like to do is to identify patterns, types of sounds, like "people talking", "loud, compressed rock music", etc.
If I understand correctly, maybe the keyword you're after is "automatic music classification"? (to which you could add e.g. "machine learning" "pure data" etc.). In this case there is loads of stuff... A good starting point (other than google) could be: http://www.ismir.net/society.html
Hope this helps. Lorenzo.
Is that feasible ? I found this library on the web : https://github.com/Cycling74/ml-lib But I have no clue how to use it.
Do you have any suggestions on where to start ? Can I feed it sound files ? Or do I need to extract some "indicators" from it (loudness, spectrum, or something) ?
Thanks in advance for your help !
Cheers,
Pierre.
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Hi all,
To answer the OP’s question: yes it is possible to “do machine learning with sound” and yes you can use ml.lib and Pd for that.
I would suggest using the upstream version of ml.lib, the version on the Cycling74 GitHub is a fork. Here’s the upstream: https://github.com/cmuartfab/ml-lib
One of the problems with ml.lib is the documentation is very poor, we are addressing this, and soon there will be a full set of help files and possibly some examples. For now, sending an object the “help” message, you might be able to figure things out. Reading our NIME paper may also help: https://nime2015.lsu.edu/proceedings/201/0201-paper.pdf
Machine learning is a very broad field, and in terms of “where to start” you might want to look at classification problems such as “out of a set of N known classes of sound, which one most closely matches sound X”. This is a well-studied problem, and you might want to start with a paper like this one: http://www.music.mcgill.ca/~ich/research/icmc00/icmc00.timbre.pdf
It would be a useful exercise to replicate the Fujinaga MacMillan experiment (which did in fact originally use Pd) using ml.knn, or indeed their original knn external, which is still available in the Pd svn.
Good luck!
Jamie
On 21 Jul 2016, at 13:42, Thomas Grill gr@grrrr.org wrote:
Please note that most applications of neural nets are non-realtime, e.g. not in the same domain as Pure Data. The evaluation of neural networks can be, but the training never is. best, Thomas
Am 21.07.2016 um 14:37 schrieb Lorenzo Sutton lorenzofsutton@gmail.com:
On 21/07/2016 12:08, Pierre Massat wrote:
Dear List,
I did a little bit of machine learning with neural network when I was in school, and I'd like to try it on sounds. What I'd like to do is to identify patterns, types of sounds, like "people talking", "loud, compressed rock music", etc.
If I understand correctly, maybe the keyword you're after is "automatic music classification"? (to which you could add e.g. "machine learning" "pure data" etc.). In this case there is loads of stuff... A good starting point (other than google) could be: http://www.ismir.net/society.html
Hope this helps. Lorenzo.
Is that feasible ? I found this library on the web : https://github.com/Cycling74/ml-lib But I have no clue how to use it.
Do you have any suggestions on where to start ? Can I feed it sound files ? Or do I need to extract some "indicators" from it (loudness, spectrum, or something) ?
Thanks in advance for your help !
Cheers,
Pierre.
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Hi all,
Thanks a lot for all these hints.
Cheers,
Pierre.
2016-07-21 15:18 GMT+02:00 James Bullock Jamie.Bullock@bcu.ac.uk:
Hi all,
To answer the OP’s question: yes it is possible to “do machine learning with sound” and yes you can use ml.lib and Pd for that.
I would suggest using the upstream version of ml.lib, the version on the Cycling74 GitHub is a fork. Here’s the upstream: https://github.com/cmuartfab/ml-lib
One of the problems with ml.lib is the documentation is very poor, we are addressing this, and soon there will be a full set of help files and possibly some examples. For now, sending an object the “help” message, you might be able to figure things out. Reading our NIME paper may also help: https://nime2015.lsu.edu/proceedings/201/0201-paper.pdf
Machine learning is a very broad field, and in terms of “where to start” you might want to look at classification problems such as “out of a set of N known classes of sound, which one most closely matches sound X”. This is a well-studied problem, and you might want to start with a paper like this one: http://www.music.mcgill.ca/~ich/research/icmc00/icmc00.timbre.pdf
It would be a useful exercise to replicate the Fujinaga MacMillan experiment (which did in fact originally use Pd) using ml.knn, or indeed their original knn external, which is still available in the Pd svn.
Good luck!
Jamie
On 21 Jul 2016, at 13:42, Thomas Grill gr@grrrr.org wrote:
Please note that most applications of neural nets are non-realtime, e.g.
not in the same domain as Pure Data.
The evaluation of neural networks can be, but the training never is. best, Thomas
Am 21.07.2016 um 14:37 schrieb Lorenzo Sutton <lorenzofsutton@gmail.com
:
On 21/07/2016 12:08, Pierre Massat wrote:
Dear List,
I did a little bit of machine learning with neural network when I was
in
school, and I'd like to try it on sounds. What I'd like to do is to identify patterns, types of sounds, like "people talking", "loud, compressed rock music", etc.
If I understand correctly, maybe the keyword you're after is "automatic
music classification"? (to which you could add e.g. "machine learning" "pure data" etc.).
In this case there is loads of stuff... A good starting point (other
than google) could be: http://www.ismir.net/society.html
Hope this helps. Lorenzo.
Is that feasible ? I found this library on the web : https://github.com/Cycling74/ml-lib But I have no clue how to use it.
Do you have any suggestions on where to start ? Can I feed it sound files ? Or do I need to extract some "indicators" from it (loudness, spectrum, or something) ?
Thanks in advance for your help !
Cheers,
Pierre.
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management ->
https://lists.puredata.info/listinfo/pd-list
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management ->
https://lists.puredata.info/listinfo/pd-list
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management ->
https://lists.puredata.info/listinfo/pd-list
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Hi Jamie,
Compiled on OSX fine with normal *.pd_darwin files. How does one install? I assume libs need to access things in other directories (DerovedData, etc) that isn't apparent to me.
Opening help patch for ml.svm in build -> macosx -> build -> Development just produces "ml.svm... couldn't create" error...
Thanks! m
On Thu, Jul 21, 2016 at 9:18 AM, James Bullock Jamie.Bullock@bcu.ac.uk wrote:
Hi all,
To answer the OP’s question: yes it is possible to “do machine learning with sound” and yes you can use ml.lib and Pd for that.
I would suggest using the upstream version of ml.lib, the version on the Cycling74 GitHub is a fork. Here’s the upstream: https://github.com/cmuartfab/ml-lib
One of the problems with ml.lib is the documentation is very poor, we are addressing this, and soon there will be a full set of help files and possibly some examples. For now, sending an object the “help” message, you might be able to figure things out. Reading our NIME paper may also help: https://nime2015.lsu.edu/proceedings/201/0201-paper.pdf
Machine learning is a very broad field, and in terms of “where to start” you might want to look at classification problems such as “out of a set of N known classes of sound, which one most closely matches sound X”. This is a well-studied problem, and you might want to start with a paper like this one: http://www.music.mcgill.ca/~ich/research/icmc00/icmc00.timbre.pdf
It would be a useful exercise to replicate the Fujinaga MacMillan experiment (which did in fact originally use Pd) using ml.knn, or indeed their original knn external, which is still available in the Pd svn.
Good luck!
Jamie
On 21 Jul 2016, at 13:42, Thomas Grill gr@grrrr.org wrote:
Please note that most applications of neural nets are non-realtime, e.g.
not in the same domain as Pure Data.
The evaluation of neural networks can be, but the training never is. best, Thomas
Am 21.07.2016 um 14:37 schrieb Lorenzo Sutton <lorenzofsutton@gmail.com
:
On 21/07/2016 12:08, Pierre Massat wrote:
Dear List,
I did a little bit of machine learning with neural network when I was
in
school, and I'd like to try it on sounds. What I'd like to do is to identify patterns, types of sounds, like "people talking", "loud, compressed rock music", etc.
If I understand correctly, maybe the keyword you're after is "automatic
music classification"? (to which you could add e.g. "machine learning" "pure data" etc.).
In this case there is loads of stuff... A good starting point (other
than google) could be: http://www.ismir.net/society.html
Hope this helps. Lorenzo.
Is that feasible ? I found this library on the web : https://github.com/Cycling74/ml-lib But I have no clue how to use it.
Do you have any suggestions on where to start ? Can I feed it sound files ? Or do I need to extract some "indicators" from it (loudness, spectrum, or something) ?
Thanks in advance for your help !
Cheers,
Pierre.
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management ->
https://lists.puredata.info/listinfo/pd-list
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management ->
https://lists.puredata.info/listinfo/pd-list
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management ->
https://lists.puredata.info/listinfo/pd-list
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
We use a combo of modified mfcc and k-means at Mogees to get over 95% accuracy of percussion sounds.
To add a few words to what Jamie says, on what I've learned about working in this domain;
- It really helps to understand your problem and the possible
ways it can be assisted by ML. Is real-time training needed? Or do you have lots of off-line time for your ML to "think"?
- How many training examples will you use, and where will
they come from? How diverse/typical are they?
- Will you supervise the ML (give clues to steer it) or allow
it to make up its own mind about classes/clusters (unsupervised)?
- What kind of output are you hoping for, prediction, regression
a definite match, or a set of probabilities, or a vector of distances from possible matches?
- How does signal time figure in your problem? Do you want
pitch or duration independence?
- How will you segment and arrange data relative to block
boundaries and the size of any transform (fft/wavelet)? Tiny variations can lead to big differences. Will you zero pad to remove junk? Will you use windows/envelopes to soften edges?
- What do you know about the source and structure of the sounds
to be processed? Are they;
i) 'Samples' with identical byte patterns? ii) Never heard before? iii) Known structure, segments, chunks. eg. speech? iv) Highly structured 'samples', hashable, eg. MIR v) Largely similar, seeking a specific structural variation? vi) Transient, sustained, harmonic? Or complex evolution?
Machine learning right now is a set of quite specialised building blocks. Each of the above shapes of problem may suggest substantially different choices of components and configuration.
Pre-processing, like shelf EQ and compression can make a _huge_ difference to the quality and reliability of results.
I havent used the ml.lib before , but the idea of having a load of ML components to play with in Pd is really attractive and I'm sure you will have tons of fun!
cheers, Andy
-
Andy Farnell -
James Bullock -
Lorenzo Sutton -
me.grimm -
Pierre Massat -
Thomas Grill