commit 71951fe476e520054908e4c53762d5d724052dda Author: Valentin Boulanger Date: Mon Jan 16 20:36:17 2023 +0100 Ajout des sources diff --git a/Emmy.pmdl b/Emmy.pmdl new file mode 100644 index 0000000..0d48a4f Binary files /dev/null and b/Emmy.pmdl differ diff --git a/README.md b/README.md new file mode 100644 index 0000000..75f569f --- /dev/null +++ b/README.md @@ -0,0 +1,197 @@ +# Snowboy Hotword Detection + +by [KITT.AI](http://kitt.ai). + +[Home Page](https://snowboy.kitt.ai) + +[Full Documentation](https://snowboy.kitt.ai/docs) + + +Version: 1.0.4 (7/13/2016) + +Snowboy is a customizable hotword detection engine for you to create your own +hotword like "OK Google" or "Alexa". It is powered by deep neural networks and +has the following properties: + +* **highly customizable**: you can freely define your own magic phrase here – +let it be “open sesame”, “garage door open”, or “hello dreamhouse”, you name it. + +* **always listening** but protects your privacy: Snowboy does not use Internet +and does *not* stream your voice to the cloud. + +* light-weight and **embedded**: it even runs on a Raspberry Pi and consumes +less than 10% CPU on the weakest Pi (single-core 700MHz ARMv6). + +* Apache licensed! + +Currently Snowboy supports: + +* all versions of Raspberry Pi (with Raspbian based on Debian Jessie 8.0) +* 64bit Mac OS X +* 64bit Ubuntu (12.04 and 14.04) +* iOS +* Android + +It ships in the form of a **C++ library** with language-dependent wrappers +generated by SWIG. We welcome wrappers for new languages -- feel free to send a +pull request! + +If you want support on other hardware/OS, please send your request to +[snowboy@kitt.ai](mailto:snowboy.kitt.ai) + +## Precompiled Binaries with Python Demo +* 64 bit Ubuntu [12.04](https://s3-us-west-2.amazonaws.com/snowboy/snowboy-releases/ubuntu1204-x86_64-1.0.4.tar.bz2) + / [14.04](https://s3-us-west-2.amazonaws.com/snowboy/snowboy-releases/ubuntu1404-x86_64-1.0.4.tar.bz2) +* [MacOS X](https://s3-us-west-2.amazonaws.com/snowboy/snowboy-releases/osx-x86_64-1.0.4.tar.bz2) +* Raspberry Pi with Raspbian 8.0, all versions + ([1/2/3/Zero](https://s3-us-west-2.amazonaws.com/snowboy/snowboy-releases/rpi-arm-raspbian-8.0-1.0.4.tar.bz2)) + +If you want to compile a version against your own environment/language, read on. + +## Dependencies + +Snowboy's Python wrapper uses PortAudio to access your device's microphone. + +### Mac OS X + +`brew` install `swig`, `sox`, `portaudio` and its Python binding `pyaudio`: + + brew install swig portaudio sox + pip install pyaudio + +If you don't have Homebrew installed, please download it [here](http://brew.sh/). If you don't have `pip`, you can install it [here](https://pip.pypa.io/en/stable/installing/). + +Make sure that you can record audio with your microphone: + + rec t.wav + +### Ubuntu/Raspberry Pi + +First `apt-get` install `swig`, `sox`, `portaudio` and its Python binding `pyaudio`: + + sudo apt-get install swig3.0 python-pyaudio python3-pyaudio sox + pip install pyaudio + +Then install the `atlas` matrix computing library: + + sudo apt-get install libatlas-base-dev + +Make sure that you can record audio with your microphone: + + rec t.wav + +If you need extra setup on your audio (especially on a Raspberry Pi), please see the [full documentation](https://snowboy.kitt.ai/docs). + +## Compile a Python Wrapper + + cd swig/Python + make + +SWIG will generate a `_snowboydetect.so` file and a simple (but hard-to-read) python wrapper `snowboydetect.py`. We have provided a higher level python wrapper `snowboydecoder.py` on top of that. + +Feel free to adapt the `Makefile` in `swig/Python` to your own system's setting if you cannot `make` it. + + +## Compile an iOS Wrapper + +Using Snowboy library in Objective-C does not really require a wrapper. It is basically the same as using C++ library in Objective-C. We have compiled a "fat" static library for iOS devices, see the library here `lib/ios/libsnowboy-detect.a`. + +To initialize Snowboy detector in Objective-C: + + snowboy::SnowboyDetect* snowboyDetector = new snowboy::SnowboyDetect( + std::string([[[NSBundle mainBundle]pathForResource:@"common" ofType:@"res"] UTF8String]), + std::string([[[NSBundle mainBundle]pathForResource:@"snowboy" ofType:@"umdl"] UTF8String])); + snowboyDetector->SetSensitivity("0.45"); // Sensitivity for each hotword + snowboyDetector->SetAudioGain(2.0); // Audio gain for detection + +To run hotword detection in Objective-C: + + int result = snowboyDetector->RunDetection(buffer[0], bufferSize); // buffer[0] is a float array + +You may want to play with the frequency of the calls to `RunDetection()`, which controls the CPU usage and the detection latency. + +## Compile an Android Wrapper + + cd swig/Android + # Make sure you set up the NDKROOT variable in Makefile before you run. + # We have only tested with NDK version r11c. + make + +Using Snowboy library on Android devices is a little bit tricky. We have only tested with NDK version r11c. We do not support r12 yet because of the removal of armeabi-v7a-hard ABI in r12. We have compiled Snowboy using Android's cross-compilation toolchain for ARMV7 architecture, see the library here `lib/android/armv7a/libsnowboy-detect.a`. We then use SWIG to generate the Java wrapper, and use Android's cross-compilation toolchain to generate the corresponding JNI libraries. After running `make`, two directories will be created: `java` and `jniLibs`. Copy these two directories to your Android app directory (e.g., `app/src/main/`) and you should be able to call Snowboy funcitons within Java. + +To initialize Snowboy detector in Java: + + # Assume you put the model related files under /sdcard/snowboy/ + SnowboyDetect snowboyDetector = new SnowboyDetect("/sdcard/snowboy/common.res", + "/sdcard/snowboy/snowboy.umdl"); + snowboyDetector.SetSensitivity("0.45"); // Sensitivity for each hotword + snowboyDetector.SetAudioGain(2.0); // Audio gain for detection + +To run hotword detection in Java: + + int result = snowboyDetector.RunDetection(buffer, buffer.length); // buffer is a short array. + +You may want to play with the frequency of the calls to `RunDetection()`, which controls the CPU usage and the detection latency. + +## Quick Start for Python Demo + +Go to the `examples/Python` folder and open your python console: + + In [1]: import snowboydecoder + + In [2]: def detected_callback(): + ....: print "hotword detected" + ....: + + In [3]: detector = snowboydecoder.HotwordDetector("resources/snowboy.umdl", sensitivity=0.5, audio_gain=1) + + In [4]: detector.start(detected_callback) + +Then speak "snowboy" to your microphone to see whetheer Snowboy detects you. + +The `snowboy.umdl` file is a "universal" model that detect different people speaking "snowboy". If you want other hotwords, please go to [snowboy.kitt.ai](https://snowboy.kitt.ai) to record, train and downloand your own personal model (a `.pmdl` file). + +When `sensitiviy` is higher, the hotword gets more easily triggered. But you might get more false alarms. + +`audio_gain` controls whether to increase (>1) or decrease (<1) input volume. + +Two demo files `demo.py` and `demo2.py` are provided to show more usages. + +Note: if you see the following error: + + TypeError: __init__() got an unexpected keyword argument 'model_str' + +You are probably using an old version of SWIG. Please upgrade. We have tested with SWIG version 3.0.7 and 3.0.8. + +## Advanced Usages & Demos + +See [Full Documentation](https://snowboy.kitt.ai/docs). + +## Change Log + +**v1.0.4, 7/13/2016** + +* Updated universal `snowboy.umdl` model to make it more robust. +* Various improvements to speed up the detection. +* Bug fixes. + +**v1.0.3, 6/4/2016** + +* Updated universal `snowboy.umdl` model to make it more robust in non-speech environment. +* Fixed bug when using float as input data. +* Added library support for Android ARMV7 architecture. +* Added library for iOS. + +**v1.0.2, 5/24/2016** + +* Updated universal `snowboy.umdl` model +* added C++ examples, docs will come in next release. + +**v1.0.1, 5/16/2016** + +* VAD now returns -2 on silence, -1 on error, 0 on voice and >0 on triggered models +* added static library for Raspberry Pi in case people want to compile themselves instead of using the binary version + +**v1.0.0, 5/10/2016** + +* initial release diff --git a/__pycache__/snowboydecoder.cpython-38.pyc b/__pycache__/snowboydecoder.cpython-38.pyc new file mode 100644 index 0000000..ed837a2 Binary files /dev/null and b/__pycache__/snowboydecoder.cpython-38.pyc differ diff --git a/_snowboydetect.so b/_snowboydetect.so new file mode 100644 index 0000000..c5b12c5 Binary files /dev/null and b/_snowboydetect.so differ diff --git a/demo.py b/demo.py new file mode 100644 index 0000000..305b8b6 --- /dev/null +++ b/demo.py @@ -0,0 +1,35 @@ +import snowboydecoder +import sys +import signal + +interrupted = False + + +def signal_handler(signal, frame): + global interrupted + interrupted = True + + +def interrupt_callback(): + global interrupted + return interrupted + +if len(sys.argv) == 1: + print("Error: need to specify model name") + print("Usage: python demo.py your.model") + sys.exit(-1) + +model = sys.argv[1] + +# capture SIGINT signal, e.g., Ctrl+C +signal.signal(signal.SIGINT, signal_handler) + +detector = snowboydecoder.HotwordDetector(model, sensitivity=0.5) +print('Listening... Press Ctrl+C to exit') + +# main loop +detector.start(detected_callback=snowboydecoder.play_audio_file, + interrupt_check=interrupt_callback, + sleep_time=0.03) + +detector.terminate() diff --git a/demo2.py b/demo2.py new file mode 100644 index 0000000..489254a --- /dev/null +++ b/demo2.py @@ -0,0 +1,41 @@ +import snowboydecoder +import sys +import signal + +# Demo code for listening two hotwords at the same time + +interrupted = False + + +def signal_handler(signal, frame): + global interrupted + interrupted = True + + +def interrupt_callback(): + global interrupted + return interrupted + +if len(sys.argv) != 3: + print("Error: need to specify 2 model names") + print("Usage: python demo.py 1st.model 2nd.model") + sys.exit(-1) + +models = sys.argv[1:] + +# capture SIGINT signal, e.g., Ctrl+C +signal.signal(signal.SIGINT, signal_handler) + +sensitivity = [0.5]*len(models) +detector = snowboydecoder.HotwordDetector(models, sensitivity=sensitivity) +callbacks = [lambda: snowboydecoder.play_audio_file(snowboydecoder.DETECT_DING), + lambda: snowboydecoder.play_audio_file(snowboydecoder.DETECT_DONG)] +print('Listening... Press Ctrl+C to exit') + +# main loop +# make sure you have the same numbers of callbacks and models +detector.start(detected_callback=callbacks, + interrupt_check=interrupt_callback, + sleep_time=0.03) + +detector.terminate() diff --git a/light.py b/light.py new file mode 100644 index 0000000..f4fd709 --- /dev/null +++ b/light.py @@ -0,0 +1,40 @@ +import RPi.GPIO as GPIO +import time + +class Light(object): + def __init__(self, port): + self.port = port + GPIO.setmode(GPIO.BCM) + GPIO.setup(self.port, GPIO.OUT) + self.on_state = GPIO.HIGH + self.off_state = not self.on_state + + def set_on(self): + GPIO.output(self.port, self.on_state) + + def set_off(self): + GPIO.output(self.port, self.off_state) + + def is_on(self): + return GPIO.input(self.port) == self.on_state + + def is_off(self): + return GPIO.input(self.port) == self.off_state + + def toggle(self): + if self.is_on(): + self.set_off() + else: + self.set_on() + + def blink(self, t=0.3): + self.set_off() + self.set_on() + time.sleep(t) + self.set_off() + +if __name__ == "__main__": + light = Light(17) + while True: + light.blink() + time.sleep(0.7) diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..68d034a --- /dev/null +++ b/requirements.txt @@ -0,0 +1 @@ +PyAudio==0.2.9 diff --git a/resources/common.res b/resources/common.res new file mode 100644 index 0000000..0e267f5 Binary files /dev/null and b/resources/common.res differ diff --git a/resources/ding.wav b/resources/ding.wav new file mode 100644 index 0000000..79346e0 Binary files /dev/null and b/resources/ding.wav differ diff --git a/resources/dong.wav b/resources/dong.wav new file mode 100644 index 0000000..426596b Binary files /dev/null and b/resources/dong.wav differ diff --git a/resources/snowboy.umdl b/resources/snowboy.umdl new file mode 100644 index 0000000..b952ca6 Binary files /dev/null and b/resources/snowboy.umdl differ diff --git a/snowboydecoder.py b/snowboydecoder.py new file mode 100644 index 0000000..4ff20c9 --- /dev/null +++ b/snowboydecoder.py @@ -0,0 +1,185 @@ +#!/usr/bin/env python + +import collections +import pyaudio +import snowboydetect +import time +import wave +import os +import logging + +logging.basicConfig() +logger = logging.getLogger("snowboy") +logger.setLevel(logging.INFO) +TOP_DIR = os.path.dirname(os.path.abspath(__file__)) + +RESOURCE_FILE = os.path.join(TOP_DIR, "resources/common.res") +DETECT_DING = os.path.join(TOP_DIR, "resources/ding.wav") +DETECT_DONG = os.path.join(TOP_DIR, "resources/dong.wav") + + +class RingBuffer(object): + """Ring buffer to hold audio from PortAudio""" + def __init__(self, size = 4096): + self._buf = collections.deque(maxlen=size) + + def extend(self, data): + """Adds data to the end of buffer""" + self._buf.extend(data) + + def get(self): + """Retrieves data from the beginning of buffer and clears it""" + tmp = ''.join(self._buf) + self._buf.clear() + return tmp + + +def play_audio_file(fname=DETECT_DING): + """Simple callback function to play a wave file. By default it plays + a Ding sound. + + :param str fname: wave file name + :return: None + """ + ding_wav = wave.open(fname, 'rb') + ding_data = ding_wav.readframes(ding_wav.getnframes()) + audio = pyaudio.PyAudio() + stream_out = audio.open( + format=audio.get_format_from_width(ding_wav.getsampwidth()), + channels=ding_wav.getnchannels(), + rate=ding_wav.getframerate(), input=False, output=True) + stream_out.start_stream() + stream_out.write(ding_data) + time.sleep(0.2) + stream_out.stop_stream() + stream_out.close() + audio.terminate() + + +class HotwordDetector(object): + """ + Snowboy decoder to detect whether a keyword specified by `decoder_model` + exists in a microphone input stream. + + :param decoder_model: decoder model file path, a string or a list of strings + :param resource: resource file path. + :param sensitivity: decoder sensitivity, a float of a list of floats. + The bigger the value, the more senstive the + decoder. If an empty list is provided, then the + default sensitivity in the model will be used. + :param audio_gain: multiply input volume by this factor. + """ + def __init__(self, decoder_model, + resource=RESOURCE_FILE, + sensitivity=[], + audio_gain=1): + + def audio_callback(in_data, frame_count, time_info, status): + self.ring_buffer.extend(in_data) + play_data = chr(0) * len(in_data) + return play_data, pyaudio.paContinue + + tm = type(decoder_model) + ts = type(sensitivity) + if tm is not list: + decoder_model = [decoder_model] + if ts is not list: + sensitivity = [sensitivity] + model_str = ",".join(decoder_model) + + self.detector = snowboydetect.SnowboyDetect( + resource_filename=resource, model_str=model_str) + self.detector.SetAudioGain(audio_gain) + self.num_hotwords = self.detector.NumHotwords() + + if len(decoder_model) > 1 and len(sensitivity) == 1: + sensitivity = sensitivity*self.num_hotwords + if len(sensitivity) != 0: + assert self.num_hotwords == len(sensitivity), \ + "number of hotwords in decoder_model (%d) and sensitivity " \ + "(%d) does not match" % (self.num_hotwords, len(sensitivity)) + sensitivity_str = ",".join([str(t) for t in sensitivity]) + if len(sensitivity) != 0: + self.detector.SetSensitivity(sensitivity_str); + + self.ring_buffer = RingBuffer( + self.detector.NumChannels() * self.detector.SampleRate() * 5) + self.audio = pyaudio.PyAudio() + self.stream_in = self.audio.open( + input=True, output=False, + format=self.audio.get_format_from_width( + self.detector.BitsPerSample() / 8), + channels=self.detector.NumChannels(), + rate=self.detector.SampleRate(), + frames_per_buffer=2048, + stream_callback=audio_callback) + + + def start(self, detected_callback=play_audio_file, + interrupt_check=lambda: False, + sleep_time=0.03): + """ + Start the voice detector. For every `sleep_time` second it checks the + audio buffer for triggering keywords. If detected, then call + corresponding function in `detected_callback`, which can be a single + function (single model) or a list of callback functions (multiple + models). Every loop it also calls `interrupt_check` -- if it returns + True, then breaks from the loop and return. + + :param detected_callback: a function or list of functions. The number of + items must match the number of models in + `decoder_model`. + :param interrupt_check: a function that returns True if the main loop + needs to stop. + :param float sleep_time: how much time in second every loop waits. + :return: None + """ + if interrupt_check(): + logger.debug("detect voice return") + return + + tc = type(detected_callback) + if tc is not list: + detected_callback = [detected_callback] + if len(detected_callback) == 1 and self.num_hotwords > 1: + detected_callback *= self.num_hotwords + + assert self.num_hotwords == len(detected_callback), \ + "Error: hotwords in your models (%d) do not match the number of " \ + "callbacks (%d)" % (self.num_hotwords, len(detected_callback)) + + logger.debug("detecting...") + + while True: + if interrupt_check(): + logger.debug("detect voice break") + break + data = self.ring_buffer.get() + if len(data) == 0: + time.sleep(sleep_time) + continue + + ans = self.detector.RunDetection(data) + if ans == -1: + logger.warning("Error initializing streams or reading audio data") + elif ans == -2: + logger.debug("Silence") + elif ans > 0: + message = "Keyword " + str(ans) + " detected at time: " + message += time.strftime("%Y-%m-%d %H:%M:%S", + time.localtime(time.time())) + logger.info(message) + callback = detected_callback[ans-1] + if callback is not None: + callback() + + logger.debug("finished.") + + def terminate(self): + """ + Terminate audio stream. Users cannot call start() again to detect. + :return: None + """ + self.stream_in.stop_stream() + self.stream_in.close() + self.audio.terminate() diff --git a/snowboydetect.py b/snowboydetect.py new file mode 100644 index 0000000..b9b9b45 --- /dev/null +++ b/snowboydetect.py @@ -0,0 +1,143 @@ +# This file was automatically generated by SWIG (http://www.swig.org). +# Version 3.0.7 +# +# Do not make changes to this file unless you know what you are doing--modify +# the SWIG interface file instead. + + + + + +from sys import version_info +if version_info >= (2, 6, 0): + def swig_import_helper(): + from os.path import dirname + import imp + fp = None + try: + fp, pathname, description = imp.find_module('_snowboydetect', [dirname(__file__)]) + except ImportError: + import _snowboydetect + return _snowboydetect + if fp is not None: + try: + _mod = imp.load_module('_snowboydetect', fp, pathname, description) + finally: + fp.close() + return _mod + _snowboydetect = swig_import_helper() + del swig_import_helper +else: + import _snowboydetect +del version_info +try: + _swig_property = property +except NameError: + pass # Python < 2.2 doesn't have 'property'. + + +def _swig_setattr_nondynamic(self, class_type, name, value, static=1): + if (name == "thisown"): + return self.this.own(value) + if (name == "this"): + if type(value).__name__ == 'SwigPyObject': + self.__dict__[name] = value + return + method = class_type.__swig_setmethods__.get(name, None) + if method: + return method(self, value) + if (not static): + if _newclass: + object.__setattr__(self, name, value) + else: + self.__dict__[name] = value + else: + raise AttributeError("You cannot add attributes to %s" % self) + + +def _swig_setattr(self, class_type, name, value): + return _swig_setattr_nondynamic(self, class_type, name, value, 0) + + +def _swig_getattr_nondynamic(self, class_type, name, static=1): + if (name == "thisown"): + return self.this.own() + method = class_type.__swig_getmethods__.get(name, None) + if method: + return method(self) + if (not static): + return object.__getattr__(self, name) + else: + raise AttributeError(name) + +def _swig_getattr(self, class_type, name): + return _swig_getattr_nondynamic(self, class_type, name, 0) + + +def _swig_repr(self): + try: + strthis = "proxy of " + self.this.__repr__() + except: + strthis = "" + return "<%s.%s; %s >" % (self.__class__.__module__, self.__class__.__name__, strthis,) + +try: + _object = object + _newclass = 1 +except AttributeError: + class _object: + pass + _newclass = 0 + + +class SnowboyDetect(_object): + __swig_setmethods__ = {} + __setattr__ = lambda self, name, value: _swig_setattr(self, SnowboyDetect, name, value) + __swig_getmethods__ = {} + __getattr__ = lambda self, name: _swig_getattr(self, SnowboyDetect, name) + __repr__ = _swig_repr + + def __init__(self, resource_filename, model_str): + this = _snowboydetect.new_SnowboyDetect(resource_filename, model_str) + try: + self.this.append(this) + except: + self.this = this + + def Reset(self): + return _snowboydetect.SnowboyDetect_Reset(self) + + def RunDetection(self, *args): + return _snowboydetect.SnowboyDetect_RunDetection(self, *args) + + def SetSensitivity(self, sensitivity_str): + return _snowboydetect.SnowboyDetect_SetSensitivity(self, sensitivity_str) + + def GetSensitivity(self): + return _snowboydetect.SnowboyDetect_GetSensitivity(self) + + def SetAudioGain(self, audio_gain): + return _snowboydetect.SnowboyDetect_SetAudioGain(self, audio_gain) + + def UpdateModel(self): + return _snowboydetect.SnowboyDetect_UpdateModel(self) + + def NumHotwords(self): + return _snowboydetect.SnowboyDetect_NumHotwords(self) + + def SampleRate(self): + return _snowboydetect.SnowboyDetect_SampleRate(self) + + def NumChannels(self): + return _snowboydetect.SnowboyDetect_NumChannels(self) + + def BitsPerSample(self): + return _snowboydetect.SnowboyDetect_BitsPerSample(self) + __swig_destroy__ = _snowboydetect.delete_SnowboyDetect + __del__ = lambda self: None +SnowboyDetect_swigregister = _snowboydetect.SnowboyDetect_swigregister +SnowboyDetect_swigregister(SnowboyDetect) + +# This file is compatible with both classic and new-style classes. + + diff --git a/version b/version new file mode 100644 index 0000000..ee90284 --- /dev/null +++ b/version @@ -0,0 +1 @@ +1.0.4