Voice Controling Sage? The future today?

[silkshadow]

Imagine being in your home and able to answer all phones, turn on and off lights/appliances, use VoIP for free calls anywhere in the world, select and play music, sit in front and control any HTPC/TV from anywhere and totally hands-free using only your voice.

I want to try and get the above working . I've been using my headset to access VoIP services for a few months now (that stuff is awesome). I'm planning on turning my soon-to-be-assembled (or reassembled) Sage server into my VoIP server as well so I can receive calls via VoIP. That part I think I got figured out. Then I discovered: http://www.x10europe.com/ for 220v X10 devices and voice control for X10 has been in existence for awhile now and I think that works well. So with all that, it seems a short step to be able to voice control my HTPC functions on this box as well.

The thing is I have no idea how to put all that together. I mean, first I'm going to need good voice recognition software that can control any piece of software. Then I'm going to need to be able to cover my home in bluetooth so I can roam freely. Of course, then I am going to have to get all the software and hardware to work with each other. I figure Girder can be used, somehow, to make the last step doable though.

So my questions are:

1) Is this even possible?

2) Can anyone recommend a good voice recognition package that will be able to control most software (especially Girder, I guess)?

2) How do I cover my home in bluetooth? I think it should not be that bad physically. I live in a condo so it’s all on one floor. I've got a few bluetooth headsets and a bunch of USB dongles. What I am thinking is that I can, one, stick bluetooth dongles into computers spaced appropriately. This shouldn't be too bad as 4 key spots with class 1 bluetooth dongles should do it. Two of which can be handled by the HTPC server itself and the HTPC client. For the other spots I have the computers but leaving them on all the time would be quite a waste. Alternatively, I can try one or two of those bluetooth access points. The thing is they seem to work like a WiFi AP (giving BT devices an IP). How would that work with parings like headset-to-server and headset-to-mobile, etc?

3) Can a bluetooth headset even do all this? Right now I am using this headset as my main one: http://www.gnnetcom.com/US/EN/MainMe...ns/GN+6110.htm. It can answer my mobile and my normal hardline without any fuss and it works with my VoIP software most of the time. I have some issues but I think if I actually spent time on sorting them out they would go away. However, adding voice recognition to its functions, not sure how that will work.

I think it would be simply incredible to actually do this.

[Crashless]

Eventually (like 5 years eventually) I'm going to use Meedio Housebot to integrate a parabolic mic with girder. Basically, Housebot uses microsoft's voice control to set parameters. You can then pass those parameters to anything you want, like a sage sendmessage.

I think there's a voicecontrol plugin for girder though. It's based on the same principal.

You _might_ be able to do it with one bluetooth dongle and a big antenna, like 7+ db. but you'd have to find a dongle that allows for easy hacking of a new antenna. Checkout flexilis.com they created a bluetooth 'sniper' rifle that bluesnarfed a phone from 2 miles away. They might be able to help.

I don't know about using multiple computers for this.

Ideally you'd get a mic balancer and place a bunch of mics all over the place. The balancer would set thresholds and only cue one mic at a time, sending that signal to your computer. You'd create keywords like "computer", or "Hal" that trigger your computer to listen to the next words.

There are a million ways to do this, and I can't write them all here. Check out home automation sites for more info about this kind of setup.

[silkshadow]

Yeah, I was thinking of housebot when I was putting all this together in my head. I am a Palm adherent though so no PPCs in my home . I was thinking of this basically because I was going to integrate VoIP to pretty much replace my standard landlines and was going to use bluetooth headsets to accomplish this. I've got 4 headsets lying around so I figured I could just use them to replace my phones . A really big antena sounds like a good idea. I hadn't considered that, thanks!

A voice control plugin for Girder would pretty much take care of basic HTPC functions. Wonder how that works, I'll take a look at that. However, girder itself can't do things like schedule a recording. I was wondering if there was a good software package that could really be like a intellegent mouse replacement with voice commands. Considering how much advancement in voice recognition theres been, there should be something like that out there. Anyone have any recomendations?

[spike5884]

Originally Posted by silkshadow

Yeah, I was thinking of housebot when I was putting all this together in my head. I am a Palm adherent though so no PPCs in my home .....
A voice control plugin for Girder would pretty much take care of basic HTPC functions. Wonder how that works, I'll take a look at that. However, girder itself can't do things like schedule a recording.....

I would suggest going to the meedio site and put in a request for Palm support in HouseBot. It won't come about anytime soon, but you might not be the only one that wants it. Also, you can set-up a housebot server and not use the "remotes". Or just use the remotes on pcs. You could set-up the server to use voice reg and then control sagetv.

Girder should be able to schedule a recording using sendmessage. I think someone even has a girder plugin for sagetv.

When I first got sagetv, I went looking for software to control it and other things(x10/stereo/tv/...). It came down to girder and housebot. I went with housebot mostly because it has remote building/support included. Right now, I have two housebot servers running. One is on the sagetv server controlling lights and such. The other one is on the sagetv client controlling the stereo/tv/sagetvclient. Eventually, I will move all the housebot control to the sagetv server.

[korben_dallas]

I'd skip Bluetooth and wait about 8-12 months for Wireless USB and other Ultra WideBand radio devices to come to market.

During that waiting period, if you aren't already a programmer, I'd try to get a basic proficiency in Java or C#, because what you are wanting to do is going to require some interfacing.

[kerryp24]

Actually, this is possible (withing limits) right now.
I use a product called HAL 2000.
It's whole purpose is to allow voice control for home automation.
It also has a developers interface called HALi.
With it you can write applications that directly interact with HAL and any other application that has a form of API.
Since SAGE has the capability to programatically control it, it is feasable to do what you ask.

I'll play with some ideas over the next couple of days and if I get something working, I'll repost to this thread.

Kerry

[PGPfan]

I hate to be the skeptic here, but I think a truely workable solution is a LONG way off. I really like the idea, and have been doing voice recognition stuff with PC's for about 10 years (anyone remember Covox?).

When I was working for Microsoft, we did a lot of work trying to develope good voice recognition software. There are just SO MANY limitations to this right now. Finally CPU's have the horsepower to do an acceptable job of recognition (especially with different accents, voices, etc.) over the phone, or a headset. The main problem area you will have is with isolating background noise. This is a MUST to get right. If you are watching a show, and want to 'pause' to go to the bathroom the voice recongnition system will not be able to differentiate between show dialogue and your 'pause show' command.

About the only way currently that I know is to use a small wireless microphone and have it on your person. Even then it is a compromise in performance and recognition accuracy but is closer to the elusive 'acoustic isolation' that you get with a telephone. The downside, of course, is that you have to remember to wear the wireless mike all the time. You might as well us a remote control, when all is 'said and done'.

Sorry to be a pessimist, I just wanted to post my experience with voice recognition.

-PGPfan

[Crashless]

I seem to remember a 'star trek' like badge being posted on engadget or gizmodo some time ago. I believe it was voip based, but it was targeted at hospitals for doctors as a replacement for pagers.

I'll see if I can dig up a link. But if you could get one of those, you could just leave it next to your door and stick it on when you come home. Kinda geeky, but it might do what you want.

[PGPfan]

Crashless,

That is by Siemens, but not confirmed if it will actually become available yet.

Here it is!

-PGPfan

[silkshadow]

Wow! Now that would be awsome!! Time to email Siemens and see if this is ever comming out. I'll add my request for a palm version of housebot. Unfortunately, I'm not a programmer but I'll take a look at HAL. That sounds like it might motivate me to go take a class. PGPfan, do you think a bluetooth headset's mic could block out background noice? The new Jabra one (http://www.jabra.com/JabraCMS/NA/EN/...JabraBT800.htm) has some sort of DSP which might be helpful there. If not, then I might have to give up this idea and wait till the tech catches up.

[PGPfan]

silkshadow,

I just looked at the Jabra link. It does look cool, but I really don't see a lot of practicality in it. For example, who is supposed to use the LCD display? I'd assume that the user would have to remove the headset/ear piece to see it. It still has buttons on it, might as well use the buttons on a remote that you are already accustomed to.

As far as sound quality, it's noise "cancellation" is probably good to effectively filter out things like wind noise, etc. from a phone conversation. Unfortunately, for something like voice control I have a hunch it won't be of any benefit. Voice recognition/control is a really specialized use of hardware and software. Take a look at the telecommunications industry as an example: they have a more or less unlimited budget and it has taken a long time to get as good at voice recongnition as they do now. (and they are pretty darned good now)

This quality came at a substancial cost but if you look at it in a simple way it works like this: A telephone (from the users perspective) is a duplex communication device. From the perspective of a voice recognition system it is a menu driven 'simplex' communication device. The necessary isolation is near perfect since the microphone is so close to the mouth during speech, and the earpiece is effectively sealed off and isolated to 'broadcasting' to the ear by itself. Try to use a voice prompted menu system with a speakerphone and you will start to see a huge degradation in performance.

Ultimately, even with perfect isolation a voice recongnition system will have a VERY difficult time deciding what words/phrases are commands that need to be acted upon versus what might be the user holding a casual conversation with someone. The way around this might be for example to use a 'push to talk' switch when you wish to speak a command. This will work, but it isn't very practical (step 1-push button, step 2 say command) versus when using a remote control as your control interface (step 1-push button, no step 2).

Sorry if it is raining on your parade. I truly wish this type of thing worked in the real world also. That damn dream of StarTrek has an unending lure for me!

-PGPfan

[Gog]

I work for one of the majors in voice recognition. Our software is used mostly in telephony settings, including applications that are used mostly with cell phones.

First, speaker independant recognition is really very good as long as you have decent grammars. That means if you want to do free speach it's going to suck But for home automation the number of commands is limited so that shouldn't be a problem (and "limited" is relative. I've seen systems work well with 100k entries in the grammar)

The acoustic models work pretty well with handsfree and cell phones so with a mic relatively close to the speaker (like a headset or the siemens gizmo) I don't think it would be that bad. I have absolutely no experience with micing a whole room though so that part I don't know. And if you want audio feedback (like a Text-To-Speech voice telling you "Yes Dave I'm changing channel now" then you get in the echo can of worms)

The "push to talk" option is the easiest to work with but you could also consider a "hotword" approach. Like Star trek who always say "computer" before giving a command, the system could wake up to a given word.

Building something like this has been a project for years and I think the technology is userfriendly enough to be relatively easy. The voice interface could be all done in VXML which is about as hard as HTML and a recognized command could do a HTTP post somewhere to trigger a sendmessage or something. The problem is that our software pretty much needs a computer on it's own (a GHz CPU and 512 MB of ram would be plenty for one user) and I never had the courage to tackle the sound input problem but I think it's doable.


Gog

[Crashless]

Meedio Housebot has a plugin to use microsoft's built in voice recognition, and a user-definable grammar list. You could probably use that as your engine to control pretty much anything.

Your biggest problem will be your voice aquisition - but you already knew that.

You might be able to hide one of these in your living room if you only need things there.

But if you want true house 'open air' mics. It'll cost ya.

http://www.iautomate.com/whole_house.html

The automatic mixer is what's going to cost you. It basically sets thresholds so that only one mic input is sent to the system at a time. That way you don't get two mics echoing eachother in the system - that'll kill your accuracy.

[kerryp24]

Yes, one of the biggest problems is the, physical, voice interface.
You have a number of options. Open air accoustical microphones is costly.
But you could do the "old" Star Trek type of thing with an intercom in each room.
The button is the trigger (sensor) to HAL to determine what room the commands will come from. Turn down/off any sounds in that room and listen for commands.

HAL also will work with ANY phone in your house that the line is also connected into the server.

Also as Gog noted, you could use an attention phrase (the default technique in HAL).

You shouldn't be so quick to dismiss the current possiblities already available.
If you want to see what people are and CAN do with this check out

http://www.james.lipsit.com/home.htm

This guy is amazing and shows just what a little ingenuity can achieve.
He also has an article on implementing an open air mic system.

BTW, I work for a VERY large company and we are working with Intel on a home automated server system based on voice. ;-)

[kpsmith]

How about using somethign like a bluetooth headset or similar tech with more range. I know you have to have the dumb headset but it would be easier than mic-ing ever room in the house.

[silkshadow]

This board is amazing!! I come up with a dream voice recognition setup, that I am hopeful I can get working, and I get people who work for leaders in the field helping me out! Thanks alot!!

To be honest PGPfan, I'm not sure what the LCD accomplishes . The description of it its really for use when the headset is sitting on your desk so one can see callerID info. However, for my use its really the DSP that comes in to play. Though I would hope there might eventually be a way to push other datafeeds to it. Not at all, before I shell out a whole bunch of money on this I need to know if its even possible.

Gog, thanks for giving me some hope . Actually, it was a call to a telco's voice recognition system that gave me the idea this would be possible. I mean if I can talk into a DSTN phone and have my commands reconized, then doing the same into a bluetooth headset might work as well in my home. I would have no problem saying: "Computer, play". Actually, I would've probably set that up anyway as I'm a big trek fan. So, is there a consumer version of your company's software, or maybe a package you would recomend that might fit for me? I can give it a standalone AMD 2000+ for itself, that would be no problem as long as I can keep broadcating on that box.

Crashless, that looks like a great place to start, thanks! That way I can see if this is at all feasable. I will spend this weekend setting that up and see how it works. I'll report back next week or whenever I get it running. Actually, I would rather avoid micing my whole home because I would like this setup to work with my VoIP stuff and having people talk into open air all the time would be a bit wierd . That iautomate site looks like a good place to pick up some hardware though.

I took a look at HAL kerryp24, that looks darn cool. However, it seems all the packages come with hardware that runs on 110v. Are there 220v packages? Man, that guy has basically what I am looking to do woriking plus a ton more. I will give housebot with MS voice recognition a try. If I can get passable performance out of that for simple things, I will press on. If that doesn't work then I will give up for now .

Thanks alot!

[kerryp24]

Silkshadow

Glad you're getting a lot of good information.

HAL, itself, is independent of the A/C in your premise. If your computer can run on it, so does HAL.
Check out their home page at
http://www.automatedliving.com

It's all the other stuff you attach on that has to worry about the A/C.
HAL works (directly) with X-10 (PL), ZWave (RF), UPB, IR devices, Security panels, HVAC... if it is any kind of industry standard, HAL probably talks to it. And if not, you can always roll your own interface using HALi.

Well, have fun with all your new found knowledge. :-)

Kerry

[Gog]

Originally Posted by silkshadow

Gog, thanks for giving me some hope . Actually, it was a call to a telco's voice recognition system that gave me the idea this would be possible. I mean if I can talk into a DSTN phone and have my commands reconized, then doing the same into a bluetooth headset might work as well in my home.

The bluetooth headset would be perfect. In face I've been feeverishly thinking about this since I'm having a new house built in a few months and I'm trying to get the automation just right

So, is there a consumer version of your company's software, or maybe a package you would recomend that might fit for me? I can give it a standalone AMD 2000+ for itself, that would be no problem as long as I can keep broadcating on that box.

I know we don't sell to individuals (thanks God, I'm in support !). I'll check for distributors who might. What area are you in ? That box would do fine and I can probably help you setting it up.

Gog

[silkshadow]

@kerryp24: Thanks for that link, looks like I was at a resellers webpage and they only had HAL bundles.

@Gog: I'm in the Philippines but getting things from Singapore should not be a problem (we're a small market here so usually distributors locate in Singapore).

Thanks guys!

[Bohica]

Hey -- in that writeup of X10 automation posted earlier, the guy talks about the open air microphones and speaker system he installed. Trying to more gear this toward Sage environments, can anyone think of software that can broadcast sound from one computer to another accross the network? Essentially using remote PCs to carry central household messages? If we used our existing clients spread throughout the house, how can we broadcast or target audio responses from one system to another? Just trying to think of potential existing applications...