Frugal Glass v2

“One never notices what has been done; one can only see what remains to be done.”
-Marie Curie

I must say it’s the case with Frugal Glass. I’m unimpressed with the performance of the current version–the eyepiece assembly is heavy enough to pull my glasses sideways, it blocks vision out of one of my eyes, the casing’s rough and the weighting hurts my ear after about an hour of use–so I decided to remake it. The new display module will be using a microcontroller I picked up as a freebie, programmable through the Arduino IDE, a small OLED screen controlled over the SPI protocol, and an Inertial Measurement Unit (IMU) consisting of an accelerometer, gyroscope, and magnetometer.

I think I’ll keep the Raspberry Pi 2–the Feather isn’t powerful enough to expand the software while making the hardware more compact. Instead, the Pi will become the CPU, while the Feather will manage the heads-up graphics.

This will allow for faster, more accurate rendering. I intend to have multiple layers of rendering mainly consisting of two types: a static heads-up display, and a 3D environment balanced by an accelerometer and gyroscope breakout board. The latter will be more difficult to implement–I can’t find any free libraries for 3D rendering, so I may have to write my own, a field in which I have no experience. Maybe I’ll look at the OpenGL source?

The eyepiece is getting a significant upgrade. An OLED screen will reflect off a semi-mirror into the user’s eye, while still letting light through. The accelerometer/gyro/magnetometer will be mounted behind it, and a mini-camera designed for first-person piloting of drones will feed into a NTSC/PAL USB dongle on the Pi.

I’ll also be doing an upgrade to the Jasper conversation system. A conversation in Jasper’s current state might look like this:

USER: Jasper

JASPER: <high beep>

USER: What’s the time?

JASPER: <low beep>

JASPER: The time is 10:20 AM right now.

This has three main problems.

The first is that you have to say the keyword first, then wait for the system to start listening, which usually takes 2-3 seconds. This is fixable by constant listening for an inline mention of the keyword. To prevent accidental references, I’ll be using the Natural Language ToolKit+ to analyze the surrounding words for context.

The second is that you have to say the keyword before every command. This is fixable by setting a command chain timer that the user can manually exit by saying something, and automatically times out after 10 seconds of inactivity.

The third is that for each command, you always get the same format of answer. (for instance, “The time is <time> right now.” This is sort of fixable by using formats selected by random.choice, but that’s not the path I’ll take. I’ve been playing with Markov Chains, and by passing a blacklist and whitelist and seeding from a random.choice, I think I can create a form of Markov Chain that will generate natural-sounding language with the gist of the desired message. It will also double-check the grammar of the sentence(s) with NLTK+ before passing it to the Text-To-Speech engine. The downside is that Markov Chains need source material, and more material is better.

Leave a Reply