Gpu | Knowledge Hub

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Nov 3

A 256-expert MoE promises 32x. At batch 1 you collect 12.5x.

#hardware #llm #performance #gpu

6 min read

Alex Chen

Nov 3

Small Models Do the Chores, Big Models Give the Answer: Cost by Task Split, Not by Model Size

#llm #ai #gpu #inference

5 min read

Amaresh Pelleti

Nov 3

Ollama Docker Setup: GPU, Volumes and Compose

#ollama #docker #dockercompose #gpu

8 min read

InferenceKey for InferenceKey

Nov 3

Serving WeVisOne-VL-4B: two engines, two copies of the weights, one chess piece apart

#ai #computervision #python #gpu

5 min read

Nov 2

A 671B model reads 36GB per token and still needs 671GB of HBM

#llm #hardware #performance #gpu

7 min read

sam ros

Nov 2

Priced per TB/s, the H200 beats the H100 at 6 of 7 providers

#ai #llm #python #gpu

6 min read

InferenceKey for InferenceKey

Nov 2

Shipping Ming-Image Design: size the card by its files, and check the alpha in code

#ai #machinelearning #python #gpu

4 min read

Arsen Apostolov

Nov 2

I Ran Whisper on a 24 GB GPU for 30 Days. It Used 8.

#homelab #gpu #whisper #asr

6 min read

Gary Nagra

Nov 2

How much VRAM do popular LLMs really need at 4-bit and 8K context? Data from 26 models

#ai #llm #machinelearning #gpu

6 min read

Masahiro Morino

Nov 1

How Does Rust Code Run on a GPU? Tracing cuda-oxide and PTX

#rust #gpu #cuda #computerscience

10 min read

aixx

Nov 1

H100 NVL

#nvidia #gpu #machinelearning #hardware

7 min read

InferenceKey for InferenceKey

Nov 1

Serving a 1B math model on vLLM: the memory cap sets how many answers fit, not how fast

#ai #llm #machinelearning #gpu

4 min read

Alexey

Oct 31

Как я впихивал три модели в 6 ГБ видеопамяти

#python #machinelearning #llm #gpu

2 min read

LK Wood IV

Oct 31

LM Studio vs Ollama on an RTX 5080: The Settings Matter More Than the Wrapper

#localai #ollama #llm #gpu

5 min read

Alexey

Oct 31

How I squeezed three models into 6 GB of VRAM

#python #machinelearning #llm #gpu

6 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.