📌 Source Repositories Things are changing quickly, so to avoid confusion, here are the exact branches used for this demo: Using: auroraGPT-ANL/torchtitan @ saforem2/blendcorpus saforem2/blendcorpus @ saforem2/reorg-imports1 🏃‍♂️ Running Clone repo: git clone https://github.com/auroraGPT-ANL/torchtitan cd torchtitan...

🎯 AuroraGPT: Goals AuroraGPT: General purpose scientific LLM Broadly trained on a general corpora plus scientific {papers, texts, data} Explore pathways towards a “Scientific Assistant” model Build with international partners (RIKEN, BSC, others) Multilingual English, 日本語, French, German, Spanish Multimodal: images,...

👀 Overview 📊 Slides @ samforeman.me/talks/openskai25/training/slides 📄 HTML version: samforeman.me/talks/openskai25/training 📑 Outline Scaling: Overview Data Parallel Training Communication Why Distributed Training? Beyond Data Parallelism Additional Parallelism Strategies Large Language Models Hands On 🚀 Scaling:...

🎯 AuroraGPT: Goals AuroraGPT: General purpose scientific LLM Broadly trained on a general corpora plus scientific {papers, texts, data} Explore pathways towards a “Scientific Assistant” model Build with international partners (RIKEN, BSC, others) Multilingual English, 日本語, French, German, Spanish Multimodal: images,...

# %matplotlib inline import matplotlib_inline matplotlib_inline.backend_inline.set_matplotlib_formats('svg') import os os.environ['COLORTERM'] = 'truecolor' import lovely_tensors as lt lt.monkey_patch() lt.set_config(color=False) # automatically detect and reload local changes to modules %load_ext autoreload...

🏖️ Shell Environment Helper function to get timestamp: tstamp() { date +"%Y-%m-%d-%H%M%S" } Load frameworks module: module load frameworks Deactivate conda environment: conda deactivate Create new conda environment: ENV_PATH="/flare/datascience/foremans/miniconda/2025-06-15" conda create --prefix "${ENV_PATH}" --y...

What When 🎨 Mixing Between Distributions While Training 2025-10-06 📊 pbs-tui: TUI for PBS Job Scheduler Monitoring 2025-09-17 🍹 BlendCorpus + TorchTitan @ ALCF 2025-09-12 🏗️ Building PyTorch 2.8 from Source on Aurora 2025-06-14 🧜‍♀️ Mermaid 2025-06-02 📰 Nice Headings 2025-06-01 🚧 Frameworks Issue with numpy > 2...

flowchart LR subgraph D["`Data`"] direction TB x("`x₀`") x1("`x₁`") x2("`x₂`") end direction LR subgraph G0["`GPU0`"] direction LR subgraph N0["`NN`"] end L0["`L0`"] end subgraph G1["`GPU1`"] direction LR subgraph N1["`NN`"] end L1["`L1`"] end subgraph G2["`GPU2`"] direction LR subgraph N2["`NN`"] end L2["`L2`"] end...

What When 🏗️ Building PyTorch 2.8 from Source on Aurora 2025-06-14 🧜‍♀️ Mermaid 2025-06-02 📰 Nice Headings 2025-06-01 No matching items CitationBibTeX citation:@online{foreman2025, author = {Foreman, Sam}, title = {06}, date = {2025-06-01}, url = {https://samforeman.me/posts/2025/06/}, langid = {en} } For...

I like headings. They help organize content and make it easier to read. Inspired by my neovim config, I wanted to recreate a similar style for headings in my website. Light Theme Dark Theme Figure 1: My neovim headings in light and dark themes. I think they turned out pretty well! Heading 1 Heading 2 Heading 3...