Spam detection in the physical world
Captured source
source ↗Spam detection in the physical world | OpenAI
April 1, 2017
Spam detection in the physical world
We’ve created the world’s first Spam-detecting AI trained entirely in simulation and deployed on a physical robot.
Loading…
Share
Sim-to-real transfer
Deep learning-driven robotic systems are bottlenecked by data collection: it’s extremely costly to obtain the hundreds of thousands of images needed to train the perception system alone. It’s cheap to generate simulated data, but simulations diverge enough from reality that people typically retrain models from scratch when moving to the physical world.
We’ve shown that domain randomization, an existing idea for making detectors trained on simulated images transfer to real images, works well for cluttered scenes. The method is simple: we randomly vary colors, textures, lighting conditions, and camera settings in simulated scenes. The resulting dataset is sufficiently variable to allow a deep neural network trained on it to generalize to reality.
Randomly generated scenes. Each frame contains Spam, often hidden among distractor objects. Our Spam model is sourced from the YCB dataset.
Our implementation
The detector is a neural network based on the VGG16 architecture that predicts the precise 3-D location of Spam in simulated images. Though it has only been trained on simulated scenes, the resulting network is able to detect Spam in real images, even in the presence of never-before-seen “distractor” items arranged in random configurations.
The video below demonstrates the system in action:
Future work
In the future, we plan to extend this work to detect phishing and to defend against adversarial Spam.
If you’d like to sink your teeth into compelling applied research problems like Spam detection, consider joining us at OpenAI.
Authors
Rachel Fong, Josh Tobin, Jack Clark, Alex Ray, Jonas Schneider, Pieter Abbeel, Wojciech Zaremba
Related articles
Point-E: A system for generating 3D point clouds from complex promptsPublicationDec 16, 2022
Multimodal neurons in artificial neural networksMilestoneMar 4, 2021
CLIP: Connecting text and imagesMilestoneJan 5, 2021