In 2019, Brett Levenson transitioned from Apple to Facebook, aiming to enhance the social media platform's content moderation during a tumultuous period marked by the Cambridge Analytica scandal. Initially, he believed that advanced technology could resolve Facebook's moderation issues, but he soon discovered that the challenges were more profound than mere tech upgrades.
Levenson noted that human moderators were burdened with memorizing extensive policy documents, often poorly translated, and had a mere 30 seconds to evaluate flagged content. This led to a concerning accuracy rate of just over 50% in their decisions, which often came long after potential harm had already occurred.
Recognizing the limitations of a reactive approach, particularly in a landscape where adversarial actors are increasingly sophisticated, Levenson conceptualized "policy as code." This innovative idea aims to transform static policy guidelines into dynamic, enforceable logic. His vision culminated in the founding of Moonbounce, which recently secured $12 million in funding, co-led by Amplify Partners and StepStone Group.
Moonbounce's mission is to integrate an additional safety layer into content generation, whether from users or AI. By employing a custom-trained large language model, the company can assess policy documents, evaluate content in real-time, and respond within 300 milliseconds. Depending on the client's needs, it can either slow down content distribution for later human review or block high-risk material immediately.
The platform currently supports three key sectors: user-generated content platforms like dating apps, AI-driven companion services, and AI image generation technologies. Levenson emphasized that safety can be a competitive advantage, a shift from the traditional view of safety as an afterthought.
Notably, Tinder has implemented similar LLM-powered moderation services, achieving a tenfold improvement in detection accuracy. This advancement is crucial as AI applications face increasing scrutiny regarding their safety measures. Levenson highlighted the need for external support in enhancing safety infrastructures, as many AI companies grapple with the implications of their technologies.
Moonbounce operates as a third-party solution, focusing solely on enforcing rules in real-time, thereby avoiding the contextual overload that chatbots often face. The company is now developing a feature called "iterative steering," designed to guide conversations towards more supportive responses when harmful topics arise, rather than simply refusing to engage.
As Levenson looks to the future, he acknowledges the potential for Moonbounce to integrate with larger platforms, though he remains cautious about the implications of such an acquisition on the technology's accessibility. His commitment to ensuring that safety becomes an integral part of product development reflects a forward-thinking approach that could redefine content moderation in the AI landscape.
With Moonbounce's innovative solutions, the future of content moderation may evolve into a proactive, integrated aspect of digital interactions, enhancing both safety and user experience across platforms.
