I just got done posting a new video tutorial up on the HYPE Vimeo channel - Understanding SoundAnalyzer. It walks you through the what, why and how of the SoundAnalyzer class - the most popular part of HYPE to date.
A few folks have been asking for a video about SoundAnalyzer - so that's what I made. If you have any ideas or requests for future videos please pass them along! Your feedback will help guide both the future of this video series and HYPE as a whole.
I would also love to know how we might get some live audio in there...So far, my only idea is to use the new 10.1 Microphone access, and plug your audio in thru the mic on your computer...ideas?
Branden Hall said, on November 18th, 2009 at 9:19 am
Right now Flash isn't setup for live audio (all audio must be played through Flash in order for it to be processed). The 10.1 microphone idea that Scott metioned should work, but another option (one we're actively looking into) is to hook up Flash to OSC (open sound control) so that it can get more accurate sound analysis data from other application and huge range of other data (including talking to Arduino boards, MIDI keyboards, etc). There are currently some existing Java apps that can provide this data to Flash, but we're looking at a pure Flash solution using AIR 2.0 (as either a bridge or a hosting environment).
Thanks Brandon for your hard work!
I've been searching for ages to find a solution for more realistic visualization in flash...and at last some movement! I'm currently playing with your SoundAnalyzer - questions to follow :)
@Scott if you want some live sound (well almost) you should think about playing with Flash Media Server and Flash Media Live Encoder. Once setup you can access the binary data from both your video and audio data in real-time.
getFrequencyIndex() appears to give me almost identical data to calling ComputeSpectrum directly (simple comparison running the same audio file via 2 apps - one with SoundAnalyzer the other without). Any chance of a brief run down of what you're actually doing that's different please?
Here's a couple of examples using a 20hz - 20khz sweep.
1. http://www.strangeloopstudios.com/viz/sweep_computespectrum_nohype/ Uses my framework for visualizing audio from FMS. It just uses bare bones ComputeSpectrum. (Sometimes you'll get an error End of file or Security exception on start - wierd fms bug - just refresh until it clears).
They behave the same except for the difference in UI display.
Branden Hall said, on November 19th, 2009 at 11:50 am
Hi Doug - under the hood SoundAnalyzer is doing a bunch of things. First off it does the obvious, call SoundMixer.computeSpectrum and reading the data out of the ByteArray into something easier to work with. It also "locks" the call to computeSpectrum so that it never happens more often than the rate you've set for SoundAnalyzer (you specify this when you call start() on your instance of SoundAnalyzer - after all SoundAnalyzer is based on AbstractRhythm, code over time).
Now, when SoundAnalyzer gets the data out of computeSpectrum it doesn't just take the numbers and go. Instead, it adjusts the scale of every number so that it goes from 0-1, taking into account psychoacoustics. That is to say, we HEAR different frequencies that are played at the same numerical volume as being different in volume. To best adjust for this, I simply played a whole lot of different songs of different genres through a small app I built and collated the data for every frequency and what it's EFFECTIVE range was. SoundAnalyzer uses that effective range when it adjusts each value in the array that it generates from computeSpectrum.
This is also why you wouldn't see much difference doing a frequency sweep - each frequency is played at the same ACTUAL volume. At the end of the day, getFrequencyIndex isn't that useful because you're getting data that represents anywhere from a whole octave to 1/92nd of an octave due to how the raw data is actually distributed. This is why I added the getOctave and getFrequencyRange calls - which looks for maxima within ranges of the data. The getOctave call is already split up to look at specific indices so that it can look a full musical octave. With getFrequencyRange, you can divvy things up however you'd like.
I get it but even when playing a full spectrum audio file not a sweep and doing a comparison i get pretty much the same result. So what's the best way to use getOctave or getFrequencyRange to do a full viz across all 256 bands of a channel? Also what about stereo files?
I managed to have a play with Hype by following and tweaking the code from your video tut - this is seriously cool, i can't wait to merge hype with my own audio visualization framework!
[...] SoundAnalyzer is currently using the getFrequencyIndex() method which should spread the ByteArray data psychoacoustically based on Brandens’ collated values. See the Hype Blog Post – Understanding… [...]
Branden Hall said, on November 19th, 2009 at 9:15 pm
When you request any value from SoundAnalyzer you can specify the range of data you want. So in the case of getFrequencyIndex you pass in the range you want, the minimum value you want, and then the maximum value you want.
As for stereo data, computeSpectrum only sends back a single channel of data when run in FFT mode. So until someone can figure out how to get a custom AS3 FFT to run in real-time, it's not going to happen. This may be a case where pre-processing the data would be the best option.
I'm a bit confused by what you say regarding stereo data; FFT mode does give back 512 bands 256 for left and 256 for right as you can see here (file pans at beginning only):
do you have a nice full spectrum demo i could see please?
Branden Hall said, on November 20th, 2009 at 1:02 pm
Hmm - something very strange is going on here. I have a series of files here that absolutely only return 256 values, but now some others are giving me 512. I'll have to look into this. I may be basing my information off of some much older experiments I did with various prerelease versions of the Flash Player. I'm going to have to see if I can consolidate what's going on into a simple example where I can see both behaviors.
Either way, there definately are improvements that can be made to SoundAnalyzer - and as time permits they'll be added. You can check out all of the demos we have just by downloading HYPE - they are all in the examples directory.
Did you see Ben Stucki has updated his blog with more info and a new FrequencyAnalyzer class. From my experiments today i can confirm it works very well but i need to play more with both sets of code. Hopefully between all of us we've got it sussed! I hope so, i'll keep you posted with what i find out. cheers
Hiya! For some reason none of the SoundAnalyser examples (or my own variations using local audio) playback the animation, only the audio.
Any ideas? (Win7)
cobra kai said, on November 30th, 2009 at 6:29 am
PS- Should add that from FlashCS4, 'test movie' does display all animations.
Branden Hall said, on November 30th, 2009 at 12:33 pm
The SoundMixer class that SoundAnalyzer uses is global across all instances of the player and thus often won't work properly when multiple flash movies are running (whether in a browser or the standalone player). The standalone player seems particularly finicky. You may want to try running the SWFs in the browser properly embedded in a HTML page.
Hey guys, Thanks for all the amazing stuff in HYPE! Really digging it.
I was wondering if you might be able to clarify a few things...If I am using getMicrophone from flash 10.1, how do I get your SoundAnalyzer to compute that data? Like the users above, I am in need of getting LIVE audio into this stuff.
Thanks!
Branden Hall said, on December 13th, 2009 at 11:21 pm
Right now Flash does not allow you to access spectrum data from the microphone. We're working on a couple of different potential solutions to this, but it probably won't be available for a few months at the least.
I've been developing this application for a few months now that I am going to be taking to a few music festivals next summer, and I really want to keep it in AIR! =)
Hype's helped quite a bit, especially with COLOR! =)
Thanks again for the awesome library...If you remember to, shoot me an email if you ever find a solution that works (or realize its impossible!) ;D
