Technology

42797 readers
84 users here now

This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.


Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.


Rules:

1: All Lemmy rules apply

2: Do not post low effort posts

3: NEVER post naziped*gore stuff

4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.

5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)

6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist

7: crypto related posts, unless essential, are disallowed

founded 7 years ago
MODERATORS
1
 
 

DualPath is a system developed by DeepSeek to address the storage input and output bottleneck that slows down agentic LLM inference. When LLMs run as agents they need to repeatedly interact with their environments over many turns which builds up a massive context history stored as a KV-Cache. Most current systems split the workload into prefill engines that process new prompt tokens and decode engines that generate the actual responses. The fundamental issue is that prefill engines have to load KV-Cache directly from external persistent storage which maxes out network bandwidth on the prefill side while the storage network connections on the decode engines sit idle.

DualPath creaties a second route for the data which allows the system to load KV-Cache from storage into the idle decoding engines first. Once the data hits the decode engines it gets forwarded to the prefill engines using a fast compute network connecting the graphics processing units. It's basically a routing strategy for aggregating the storage bandwidth across all the machines and stop the prefill nodes from becoming a choke point.

A traffic manager places the KV-Cache transfers onto a lower priority virtual lane so that the actual inference communication gets majority of the bandwidth priority while data shuffling happens in the background without causing latency spikes. A dynamic scheduler then constantly monitors token counts and queue lengths to distribute the reading tasks evenly across all available hardware. In teests, DualPath improved system throughput by nearly two times compared to a standard setup. Turns out that properly balancing network traffic that was already available in the cluster makes multi-turn agent workloads dramatically faster.

2
3
4
5
6
7
8
9
10
11
 
 

What we have here is a massive reality check for the current obsession with blindly scaling up parameters to get better performance proving that you can squeeze frontier level logical reasoning into a tiny 3b parameter model. It managed to hit a score of 94.3 on the extremely difficult AIME26 math benchmark and got an 80.2 on LiveCodeBench v6 putting their incredibly small model in the exact same weight class as massive flagship models like Gemini 3 Pro.

They pulled it off using optimized post training pipeline based on their Spectrum to Signal paradigm starting with curriculum based supervised fine tuning to teach the model broad concepts before forcing it to focus on extremely hard and long reasoning problems. After that they ran multi domain reinforcement learning with a huge 64K context window to make sure the model could actually finish its long thoughts without getting artificially truncated. Another trick they used was to include a Long2Short reinforcement learning stage designed to force the model to be more token efficient in its math reasoning without losing accuracy. And tied it all together with offline self distillation to bake advanced reasoning skills into the base model.

The authors argue that the industry has been conflating two different types of artificial intelligence capabilities. Memorizing world knowledge and random facts naturally requires an expansive amount of parameters. However, pure verifiable reasoning like math and code is actually parameter dense because it is mostly just search, constraint satisfaction, and error correction. So you can tightly compress a world class reasoning engine into a tiny model without needing hundreds of billions of parameters to store random trivia. A big takeaway here is that small models aren't just cheap fallbacks for when you cannot afford massive compute and can legitimately be used for building top tier reasoning systems.

https://huggingface.co/WeiboAI/VibeThinker-3B

a version fine tuned for tool calling oh even better https://huggingface.co/Shadow0482/mythos_fast

12
13
 
 

I have such setup, but with some problems:

  • it always does full crawl (doesn't do deduplication)
  • I am unable to control crawl order (don't have enough knowledge)
14
15
16
17
18
19
20
21
 
 

With a ~40 billion usd hole (netloss) Openai keeps it´s word by staying a nonprofit company 🤣

22
23
24
 
 

Israel's government asked Meta to censor social media content about its ongoing war against Iran, according to internal documents viewed by The Intercept.

Company records show that Israel petitioned Meta to take down Facebook and Instagram posts expressing support for Iran, opposition to Israel, and even depictions of Iranian missile impacts.

The government flagged a variety of materials related to the war, including posts mourning the death of Ayatollah Khamenei following his assassination by the U.S. and Israel on the opening day of the conflict, content supportive of Iran’s retaliatory attacks, and Iranian accounts that shared military analysis and ~~propaganda~~ news sympathetic to the Iranian ~~regime~~ government’s perspective.

In some cases, Meta complied with the censorship requests, the records show, though it is unclear on what grounds. Meta maintains that it only removes content as required by law or materials that violate its speech policies.

25
view more: next ›