In 2024, 802.11bf Wi-Fi 7 Sensing with mmWave radar could see through home/business walls to monitor keyboard typing, human heartbeats and other activity. While attackers can use custom radio firmware today, WiFi 7 will bring "X-Ray Vision" to consumers, ready or not, unless nation-state spectrum regulators intervene. https://news.ycombinator.com/item?id=30172647. If a neighbor buys a Wi-Fi 7 Sensing device, they could monitor human activity through your walls.
Incidentally, it sure would be a shame if using plaster for home interiors came back into fashion, since the metal lath acts as an effective Faraday cage.
Wi-Fi 7 Sensing works at each of the different frequencies (2.4Ghz, 5Ghz, 6Ghz, 60Ghz). The lower frequencies have lower imaging resolution, longer range/penetration and can be blocked with less dense mesh.
EMF shielding materials readily available at hardware stores include insect screening metal mesh and aluminum vapor or radiant barrier. There's also drywall which includes RF shielding, but it's expensive. DIY pointers: https://mpkb.org/home/special/emf/whitezones/faradaycage
I was CTO/head of R&D at a company where we were sensing where someone
touched an object based on sounds. I was able to semi-accurately
position a finger tap on a glass phone surface. That was in 2016.
Caveats:
We used contact microphones. Taking the data out of the air from an
acoustic signal would be much harder, but not impossible in a quiet
room with some fancy DSP.
There is a symmetry such that tapping the same distance from corner A
is indistinguishable from tapping the equivalent position in corner B.
Still, for a 4 x 4 PIN entry that's useful extra info.
It depends on the physiology, finger length and nails of the operator,
and how they hold the phone. The fact that this may also be a unique
identifier of the operator should not be a surprise.
The sensors are piezo micro-discs. They come in all shapes and sizes
these days, adhesive piezo ribbons and much more. To get a good
signal you really need an op-amp or jfet stage too.
I have an extremely loud keyboard[1] and a decent quality Blue Snowball[2] microphone (right next to each other, at that), and it can't predict anything reliably. This is quite surprising, as the spacebar I could probably guess by ear myself.
I just gave it a try myself. My mic isn't as nice (Logitech C920) but my keyboard is also pretty loud (Das Keyboard Prime 13). I gave it two tries and it didn't even get within an order of magnitude on the number of keystrokes, let alone what the keystrokes are.
I'm not dunking on them. This is a really really hard problem. My comment is more that I'm not worried about this as a viable threat just yet.
Hey, author here - great to see this posted on HN again. Let me know if you have any questions.
I notice comments that the approach is not working for various setups. I recommend starting with the most simple test - using Keytap [0] and training it with just 2 keys (for example 'q' and 'p' on QWERTY keyboard). You should get nearly 100% recall rate with "Average CC" above 0.80 for each of the 2 keys. If this is not the case, then Keytap will most certainly not work for your setup.
"We show that PC keyboards, notebook keyboards, telephone and ATM pads are vulnerable to attacks based on differentiating the sound emanated by different keys. Our attack employs a neural network to recognize the key being pressed. We also investigate why different keys produce different sounds and provide hints for the design of homophonic keyboards that would be resistant to this type of attack."
ATM keypads are an extremely expensive component that you replace as a complete module, made out of a couple of chunks of very thick stainless steel and plastic with the PCB in the middle.
They are specifically designed so that all the keys sound exactly the same, and individually tested at the manufacturer. This is something they've already thought about, back in the early 1990s.
A week ago here on HN, someone posted that they had a program that automatically muted the microphone on keyboard input. That should be effective against this being used during teleconferencing.
This is very cool. Back in college, we had keyboards that were overused and of not very high quality. The wear and tear of several students pounding on them made the sounds of many keys distinguishable. The administrators had a simple password for root. It had two spaces and two number keys in it. Totally 7 letters long. I made some educated guesses by listening to the sounds while they typed it in and then used those to narrow down my brute forcing program (a C program I might add). That got me in in a few hours.
Okay, now imagine that you're remotely pair-programming with someone (over Zoom or Skype or Teams or whatever). You get to see and hear a lot of the keystrokes and the characters they emit.
Then, at some point, you or your peer are prompted to enter a password. The password field shows up as all bullets. But... can you still identify the password based on the audio feed?
Yes - but you don't even need their output. I did this a dozen or so years ago at a coworking space, where I trained it on my officemate's keyboard assuming they were typing Ruby code, and then was able to guess their passphrase pretty quickly using the trained model.
I manually fudged spacebars and enters because they're accoustically obvious, and played around with punctuation keys. Generally the timing for fingers to move from one key to the other was where I was finding the strongest signal.
I doubt the audio fidelity over a video call would be high enough to do this same thing, though. Esp. since all of those services have noise reduction stuff in place by default.
I’m guessing it doesn’t need to be all that accurate for certain use cases.
Even just knowing the length of the password, estimating which keys in the sequence are capitalized (if Shift behavior is fairly easy to pin down) and being able to pin each key down to 5 possibilities would make a 20 character password trivial to crack. Right?
Perhaps the keyboard could have an internal speaker that provides carefully crafted additional noise to make the keys sound more uniform (like each-other), or to sound more random (different each time).
However, if the counter-noise would get triggered first by a key press then it would not be able to mask the initial part of the key noise when the finger strikes the key surface before pressing it down.
Detection using a microphone would have the risk of false positives, so maybe a new key mechanism also including capacitive touch/proximity sensors would be needed.
And it would be ineffective against attacks that model key stroke patterns temporally.
This should still work on quiet keyboards--it just requires a bit more work in setting up a listening device. For example, you could hide a microphone under the keyboard or attach it to the table it's sitting on (and maybe adjust the model accordingly).
I need to break out my relay board to see if the sound of the relays clicking mitigates attacks like this: https://youtu.be/6hMOGKTudcg (see it in all its clicky glory!)
As long as the click of the relay happens fast enough--and I add some sound dampening to the keyboard (which was the opposite of what I did for that test video haha)--I bet it would render this kind of attack useless.
If you're going to go through the trouble of placing a microphone under someone's table or keyboard, you mind as well just install a keylogger. It would be much more likely to succeed given the risk.
I've had the suspicion that an attack like this has already been used in the past on a few Twitch/YouTube streamers.
They possess a large corpus of training data (heck, some of them play typeracer on-screen.), and would no doubt have a few with quite audible mechanical keyboards near decent recording setups.
Then again, good security hygiene still mitigates this. (Avoiding password re-use, using 2FA where available etc.)
You wouldn't be emitting the sounds of typing your password while using a password manager either... (Well, except for the unlock password. Also there's something to be said about clipboard implementation across different operating systems.)
Does this assume that the keyboard is qwerty, or is it able to identify typing patterns regardless of the keyboard layout? I use Dvorak and I couldn't get any of the demos to work for me, but that may just be my fault.
I skimmed the readme but a chunk of it is admittedly over my head, however I would expect something based on letter frequency and n-grams to work regardless of layout, while something that relies on the acoustics of the individual keys to be layout-dependent.
Based on the readme it seems to need training data, so assuming you train it with the same layout you want to use later to capture from. i don't see why it wouldn't work.
It does not require training data - instead it uses statistical information about the frequencies of the letters and n-grams in the English language.
and from this it should also be noted that it won't apparently be able to extract passwords, as least those which aren't "n-grams in the English language".
This could, and should be mitigated through the use of real time keyboard remappings. An image of a randomized keyboard layout is shown to the user so they can type their password using different keys. Enjin Wallet on IOS does this (obviously for reasons other than acoustics), but I have not seen it used anywhere else.
Preliminary looks into remapping keyboard inputs in real time looks annoying to make portable, but doable.
Not this specific code, but I have done accoustic keyboard analysis in the past with success. The thing I wasn't able to get working was multiple keyboards in a single room - in theory it should be doable, but the signal processing was (and still is) far beyond my rudimentary skills.
It should be possible to isolate each separate keyboard with a microphone array (but yes, signal processing...)
Which makes me wonder: With a sensitive enough microphone array, might it be possible to separate out the locations of each individual key? At the very least, it seems like it might be possible if it's coming from the right or the left side of the keyboard.
Yeah, that was the rough plan, start with a pair of mics and triangulate out - I was hoping to iterate all the way up to a pair of laser microphones on different window panes, and on paper I think it should be all doable. I'd assume all of the organizations that have budgets for things like this already have it, but it was a fun project to hack on between gigs.
> “I think political systems will use it to terrorize people,” Hinton said. Already, he believed, agencies like the N.S.A. were attempting to abuse similar technology.
> “Then why are you doing the research?” Bostrom asked.
> “I could give you the usual arguments,” Hinton said. “But the truth is that the prospect of discovery is too sweet.” He smiled awkwardly, the word hanging in the air—an echo of Oppenheimer, who famously said of the bomb, “When you see something that is technically sweet, you go ahead and do it, and you argue about what to do about it only after you have had your technical success.”
Today, gr-tempest can recreate monitor images from HDMI RF emissions, using a low-cost SDR, https://twitter.com/markopolophys/status/1459911025642414086. GNU Radio conference video: https://www.youtube.com/watch?v=QRycX0M0H4s. TEMPEST techniques have been known for decades, but only recently became accessible to pen-testers via OSS code and low-cost hardware, https://en.wikipedia.org/wiki/Tempest_%28codename%29 & https://en.wikipedia.org/wiki/Van_Eck_phreaking.
In 2024, 802.11bf Wi-Fi 7 Sensing with mmWave radar could see through home/business walls to monitor keyboard typing, human heartbeats and other activity. While attackers can use custom radio firmware today, WiFi 7 will bring "X-Ray Vision" to consumers, ready or not, unless nation-state spectrum regulators intervene. https://news.ycombinator.com/item?id=30172647. If a neighbor buys a Wi-Fi 7 Sensing device, they could monitor human activity through your walls.
Intel (2020) presentation on Wi-Fi 7, slides #19 and #20 describe Wi-Fi Sensing, https://www.intel.com/content/dam/www/public/us/en/documents...