VentureBeat

New LLM optimization technique slashes memory costs up to 75%

Researchers at the Tokyo-based startup Sakana AI have developed a new technique that enables language models to use memory more efficiently, helping enterprises cut the costs of building applications ...
TMCnet

Inception Launches Mercury 2, the Fastest Reasoning LLM - 5x Faster Than Leading Speed-Optimized LLMs, with Dramatically Lower Inference Cost

Inception, the company behind the first commercial diffusion large language models (dLLMs), today announced the launch of Mercury 2, the fastest reasoning LLM and first reasoning dLLM. Mercury 2 ...
Semiconductor Engineering

HW-SW Co-Designed System With 3 Core Optimization Pathways For Long-Context Agentic LLM Inference (Cambridge, ICL)

A new technical paper titled “Combating the Memory Walls: Optimization Pathways for Long-Context Agentic LLM Inference” was published by researchers at University of Cambridge, Imperial College London ...
InfoWorld

Multi-token prediction technique triples LLM inference speed without auxiliary draft models

With reported 3x speed gains and limited degradation in output quality, the method targets one of the biggest pain points in production AI systems: latency at scale.

Manifold-Constrained Hyper-Connections: The Architectural Breakthrough That Might Redefine LLM Training

If mHC scales the way early benchmarks suggest, it could reshape how we think about model capacity, compute budgets and the ...
Semiconductor Engineering

Detailed Study of Performance Modeling For LLM Implementations At Scale (imec)

A new technical paper titled “System-performance and cost modeling of Large Language Model training and inference” was published by researchers at imec. “Large language models (LLMs), based on ...

Taalas Launches Hardcore Chip With ‘Insane’ AI Inference Performance

Taalas has launched an AI accelerator that puts the entire AI model into silicon, delivering 1-2 orders of magnitude greater ...
Business Wire

ASC24 Finals Set for April in Shanghai: Focus on Cutting-Edge Large Language Model Inference and Seepage Simulation!

BEIJING--(BUSINESS WIRE)--On January 4th, the inaugural ceremony for the 2024 ASC Student Supercomputer Challenge (ASC24) unfolded in Beijing. With a global interest, ASC24 has garnered the ...
XDA Developers on MSN

I served a 200 billion parameter LLM from a Lenovo workstation the size of a Mac Mini

This mini PC is small and ridiculously powerful.