What once was an American college student messing around with coding in his dorm at Harvard University became the world’s most widely used social media platform. As if that isn’t enough, currently Facebook has growing ambitions beyond social media which has manifested in its recent name change - Meta - to reinvent itself as a tech company that connects people. The mission has been in the works for quite some time, as reflected by Facebook expanding its offices and hiring from all over the world.
You may be wondering why Facebook chose to expand in Pittsburgh and what goes on inside the walls of it’s District 15 building in the Strip District.
A hub for Artificial Intelligence (AI) since the 1950s and dubbed the #1 computer science program in the nation, Carnegie Mellon University (CMU) students and faculty are attractive candidates for careers with large tech corporations. Over the years, CMU has teamed up with Facebook on numerous occasions to complete projects, and in turn Carnegie Mellon has received millions in funding, allowing researchers to put aside their attempts for research funding, and focus fully on their projects.
More recently, Facebook launched their Research & AI Mentorship (FRAIM) Program which allows Pittsburgh’s brightest to get involved in research with Facebook, further strengthening the existing partnership between Facebook and CMU.
The District 15 office, which funnels in researchers from CMU, was built with the intention of expanding their Virtual Reality (VR) and Augmented Reality (AR) research. More specifically, Facebook’s mission was to utilize VR/AR to foster connection between people physically far - a pandemic-inspired project you could say. The end product of this research is still in the works, but according to Zuckerberg could hit the market as early as 2022.
However, there is a collaboration between Facebook and CMU within the field of AI that has come to fruition more rapidly: ReSkin.
Throughout this article, we will cover:
As humans, the majority of our day-to-day behaviors are largely unconscious – that is, we don’t have to think about it. We instinctively know not to touch fire, to throw out mushy fruit, that a hot head means we are ill, that ice is slippery. This ability to feel is one of the last sensory traits distinguishing humans from artificial intelligence - unless it isn’t.
Recently, after Facebook rebranded itself Meta, it announced a collaboration with CMU to work towards AI technology that can understand and interact with the world in the unique way that tactile touch allows humans to. They created ReSkin.
ReSkin is open-source touch sensing synthetic “skin.” It is 2 to 3 mm thick, stretchable, and constructed out of elastomer membrane with embedded magnetic microparticles. With the intention of making it user-friendly, CMU and Facebook researchers designed the product so as to be as easy to apply as a bandaid.
The ReSkin is composed of two working bodies: the ReSkin component and the electronic component. The ReSkin constituent is an artificial soft-material much like skin and the electronic component is a small magnetometer. The magnetometer lies beneath or immediately nearby the ReSkin and determines the magnitude of force applied and location of touch using special algorithms that detect and analyze changes in the material’s magnetic field. When the ReSkin touches something, the magnetic particles move, altering their distance from one another, and in turn creating changes in the magnetic field. This provides the magnetometer with the data it needs to make its calculations. ReSkin is manufactured at $6 per unit for an 100-unit manufacturing run.
In comparison to sound and vision, touch remains a challenging and underdeveloped field in the AI industry. It is not that others have not attempted to capitalize on this unfounded industry, but that assimilating human-like touch into electronic models is an arduous endeavor. This is in part due to the limited tactile-sensing data in the wild. The ReSkin development required 100 touches from a human just to get the minimum data necessary to analyze the changes in magnetic field associated with sense of touch, and then replicate it.
The National University of Singapore also developed an electronic skin in the lab, termed “soft-skins,” but it has a long history of being expensive and challenging to reproduce, let alone mass produce. Each sensor has to be individually calibrated, and depending on how it is used and how it ages, the calibrations can change.
Overall, this method was not cost efficient, by means of maintenance, time, or funds. ReSkin circumvents these issues by eliminating the need for electrical connection between the soft materials and measurement materials, deeming it to be easily replaceable.
As previously mentioned, ReSkin components rely on proximity rather than connection. As a result, ReSkin has sensors that autocallibrate - or, fine-tune automatically- to remain consistent throughout use and repair. No hardware or software changes are required. Taken as a whole, ReSkin offers a diverse range of versatile, scaleable, and economical tactile sensation modules unlike anything that has been created before.
The ultimate question that this announcement leads to is - why? - to which there are a myriad of answers ranging from AR/VR, industrial, agriculture, all the way to medical.
First, the conceptualization of artificial touch goes hand-in-hand with the aforementioned Meta goal of connecting people separated by distance. In theory, ReSkin could be used in Meta’s aspirations for a metaverse – a network of 3D virtual worlds where people can go to connect. As they continue to fine tune their VR headsets, ReSkin could just be the missing piece to enrich the lifelike experience that Meta is striving for in the metaverse. By incorporating touch, people may never feel lonely again, no matter how alone they are.
Another potential application of ReSkin is in prosthetics or some variation of a “smart glove” that allows someone to feel what another person is touching. Reviving someone's sense of touch could prove to be groundbreaking medicine. Along the same lines, Meta tested ReSkin’s ability to track the magnitude and direction of applied force during resting, walking, and running by placing it along the sole of a shoe along with a circuit board. They obtained this data with a dog, so imagine the potential for a podiatrist or a company specializing in customized running shoes.
Other suggestions for use have included robotics, wearables, smart-clothing, object-classification, proprioception, and robotic grasping. The research team tested the system’s amiable touch on a pack of ripe blueberries and grapes. The system performed the handling of the fruit with dexterity, implying one of it’s more innate uses. ReSkin may be superior to other robotics systems which may fall prey to squished or damaged fruit through a heavy grip.
As long as we have ideas, the ways to implement this touch system are limitless. Meta feels confident, and we agree that ReSkin is the future of AI and it’s utility knows no bounds as long as there is creativity. As long as CMU graduates continue funneling into roles at Facebook, and collaborating right in our backyard, we can expect innovation to flourish.
