Threads has entered the fediverse
Meta Engineering
by
6d ago
Threads has entered the fediverse! As part of our beta experience, now available in a few countries, Threads users aged 18+ with public profiles can now choose to share their Threads posts to other ActivityPub-compliant servers. People on those servers can now follow federated Threads profiles and see, like, reply to, and repost posts from the fediverse. We’re sharing how we’re continuing to integrate Threads with the fediverse, the technical challenges, the solutions we’ve come up with along the way, and what’s next as we move toward making Threads fully interoperable. Threads’ initial laun ..read more
Visit website
Optimizing RTC bandwidth estimation with machine learning
Meta Engineering
by
1w ago
Bandwidth estimation (BWE) and congestion control play an important role in delivering high-quality real-time communication (RTC) across Meta’s family of apps. We’ve adopted a machine learning (ML)-based approach that allows us to solve networking problems holistically across cross-layers such as BWE, network resiliency, and transport. We’re sharing our experiment results from this approach, some of the challenges we encountered during execution, and learnings for new adopters. Our existing bandwidth estimation (BWE) module at Meta is based on WebRTC’s Google Congestion Controller (GCC). We ..read more
Visit website
Better video for mobile RTC with AV1 and HD
Meta Engineering
by
1w ago
At Meta, we support real-time communication (RTC) for billions of people through our apps, including Messenger, Instagram, and WhatsApp. We’ve seen significant benefits by adopting the AV1 codec for RTC. Here’s how we are improving the RTC video quality for our apps with tools like the AV1 codec, the challenges we face, and how we mitigate those challenges. The last few decades have seen tremendous improvements in mobile phone camera quality as well as video quality for streaming video services. But if we look at real-time communication (RTC) applications, while the video quality also has im ..read more
Visit website
Logarithm: A logging engine for AI training workflows and services
Meta Engineering
by
1w ago
Systems and application logs play a key role in operations, observability, and debugging workflows at Meta. Logarithm is a hosted, serverless, multitenant service, used only internally at Meta, that consumes and indexes these logs and provides an interactive query interface to retrieve and view logs. In this post, we present the design behind Logarithm, and show how it powers AI training debugging use cases. Logarithm indexes 100+GB/s of logs in real time, and thousands of queries a second. We designed the system to support service-level guarantees on log freshness, completeness, durability ..read more
Visit website
Building Meta’s GenAI Infrastructure
Meta Engineering
by
2w ago
Marking a major investment in Meta’s AI future, we are announcing two 24k GPU clusters. We are sharing details on the hardware, network, storage, design, performance, and software that help us extract high throughput and reliability for various AI workloads. We use this cluster design for Llama 3 training. We are strongly committed to open compute and open source. We built these clusters on top of Grand Teton, OpenRack, and PyTorch and continue to push open innovation across the industry. This announcement is one step in our ambitious infrastructure roadmap. By the end of 2024, we’re aiming t ..read more
Visit website
Making messaging interoperability with third parties safe for users in Europe
Meta Engineering
by
3w ago
To comply with a new EU law, the Digital Markets Act (DMA), which comes into force on March 7th, we’ve made major changes to WhatsApp and Messenger to enable interoperability with third-party messaging services.  We’re sharing how we enabled third-party interoperability (interop) while maintaining end-to-end encryption (E2EE) and other privacy guarantees in our services as far as possible. On March 7th, a new EU law, the Digital Markets Act (DMA), comes into force. One of its requirements is that designated messaging services must let third-party messaging services become interoperable ..read more
Visit website
How DotSlash makes executable deployment simpler
Meta Engineering
by
1M ago
Andres Suarez and Michael Bolin, two software engineers at Meta, join Pascal Hartig (@passy) on the Meta Tech Podcast to discuss the ins and outs of DotSlash, a new open source tool from Meta. DotSlash takes the pain out of distributing binaries and toolchains to developers. Instead of committing large, platform-specific executables to a repository, DotSlash combines a fast Rust program with a JSON manifest prefixed with a #! to transparently fetch and execute the binary. Learn how DotSlash was built, how it’s used at Meta, and how Michael and Andres’ career trajectories lead them to create th ..read more
Visit website
Aligning Velox and Apache Arrow: Towards composable data management
Meta Engineering
by
1M ago
We’ve partnered with Voltron Data and the Arrow community to align and converge Apache Arrow with Velox, Meta’s open source execution engine. Apache Arrow 15 includes three new format layouts developed through this partnership: StringView, ListView, and Run-End-Encoding (REE). This new convergence helps Meta and the larger community build data management systems that are unified, more efficient, and composable. Meta’s Data Infrastructure teams have been rethinking how data management systems are designed. We want to make our data management systems more composable – meaning that instead of i ..read more
Visit website
Meta loves Python
Meta Engineering
by
1M ago
By now you’re already aware that Python 3.12 has been released. But did you know that several of its new features were developed by Meta? Meta engineer Pascal Hartig (@passy) is joined on the Meta Tech Podcast by Itamar Oren and Carl Meyer, two software engineers at Meta, to discuss their teams’ contributions to the latest Python release, including new hooks that allow for custom JITs like Cinder, Immortal Objects, improvements to the type system, faster comprehensions, and more. Learn how and why they built these new features for Python and how they worked with and engaged with the Python com ..read more
Visit website
Simple Precision Time Protocol at Meta
Meta Engineering
by
1M ago
While deploying Precision Time Protocol (PTP) at Meta, we’ve developed a simplified version of the protocol (Simple Precision Time Protocol – SPTP), that can offer the same level of clock synchronization as unicast PTPv2 more reliably and with fewer resources. In our own tests, SPTP boasts comparable performance to PTP, but with significant improvements in CPU, memory, and network utilization. We’ve made the source code for the SPTP client and server available on GitHub. We’ve previously spoken in great detail about how Precision Time Protocol is being deployed at Meta, including the protoco ..read more
Visit website

Follow Meta Engineering on FeedSpot

Continue with Google
Continue with Apple
OR