CURRENTLY AT NVIDIA

Jahanzeb Hashmi

Senior Architect

Resume CV Google Scholar

About

// summary
Jahanzeb Hashmi

I am a Senior Architect (HPC) at NVIDIA, where I primarily work on HW/SW co-design of key HPC/AI workloads. I lead cross-organization working groups responsible for architectural design trade-offs, performance modeling and projections, and competitive analysis of NVIDIA and competitive platforms. I work on establishing speed-of-light bounds for performance, energy efficiency, and TCO for NVIDIA's leadership class systems such as multi-node NVLINK. My work influences NVIDIA's GPU roadmap and provides actionable insights to the leadership.

Before NVIDIA, I was a Senior Research Associate at The Ohio State University, working in the Network-Based Computing Laboratory on MVAPICH2, a high-performance MPI library used across supercomputing centers worldwide. My focus was GPU-aware MPI for RDMA networks on AMD and NVIDIA hardware, hierarchical collective algorithms for scalable scientific and deep learning workloads, zero-copy data movement for sparse layouts, and performance engineering of parallel applications on multi-petaflop systems.

Technical Focus

// what I work on

HW/SW Co-design

Workload-aware architecture design trade-off across NVIDIA and competitive platforms that directly influences NVIDIA product roadmap for next-gen GPU, CPU, and networks.

Performance Engineering

Bottleneck analysis of key HPC/AI applications from chip-level microarchitecture up through full datacenter-scale systems.

Projections and Competitive Analysis

Performance modeling and projections, scaling-limiter analysis, and actionable insights leading to improved product designs.

Communication Libraries (MPI, SHMEM, NCCL)

Design and development of GPU-aware communication libraries focusing collective communications for multi-GPU systems with RDMA networks.

Experience

// 2015 — present
MAR 2021 — PRESENT

Senior High Performance Compute Architect

NVIDIA Corporation, USA
  • Model speed-of-light system performance (compute and communication) to set GPU utilization roofline targets for next-generation accelerator hardware.
  • Identify scaling limiters and co-design applications and communication libraries for maximum architecture-allowed performance and efficiency.
  • Lead the HPC performance projections working group evaluating current and next-gen accelerator architectures for key HPC and AI/LLM workloads.
  • Lead the HPC competitive analysis working group, projecting performance from kernel/chip-level analysis up to full-scale datacenter architecture.
JUN 2020 — MAR 2021

Senior Research Associate

The Ohio State University, USA
  • Designed a high-performance MPI library for next-gen HPC/cloud systems spanning multi-core CPUs and GPUs (NVIDIA, AMD).
  • Led design of a generalized hierarchical MPI collective communications framework optimized for modern CPU+GPU node architectures.
  • Led MVAPICH2-GDR, a GPU-aware MPI library for NVIDIA/AMD multi-GPU systems, from architecture through production deployment.
  • Mentored Ph.D. and M.S. students on communication runtime design for CPUs, GPUs, and high-speed interconnects.
AUG 2015 — MAY 2020

Graduate Research Associate

The Ohio State University, USA
  • Designed an adaptive, topology-aware algorithm mapping MPI processes to hardware cores based on communication patterns of AI/HPC workloads.
  • Collaborated on efficient parallelization of large-scale distributed DNN training (data- and model-parallel) across CPU/GPU systems.
  • Designed a zero-copy XPMEM-based inter-process communication layer for manycore/multi-GPU architectures.
  • Designed a data-layout caching algorithm mitigating MPI derived-datatype translation costs — Best Paper Award nominee, IPDPS '19.

Skills

// tools & stacks

GPU & Accelerator Systems

CUDA / HIPNCCL GPU-aware MPI/SHMEMNVIDIA Nsight Suite Multi-GPU/Multi-node Profiling

Performance Engineering

Roofline ModelingFLOPs / Bandwidth Analysis Interconnect Throughput

System Design

HW/SW Co-designEnd-to-end Perf/Power/TCO Optimization

Languages

CC++PythonJava

Select Publications

// peer-reviewed

For the complete list, see my Google Scholar page.

Towards Architecture-aware Hierarchical Communication Trees on Modern HPC SystemsBEST PAPER FINALIST
B. Ramesh, J. Hashmi, S. Xu, A. Shafi, M. Ghazimirsaeed, M. Bayatpour, H. Subramoni, D. K. Panda
HiPC — 28th IEEE International Conference on High Performance Computing, Data, Analytics and Data Science, 2021
GEMS: GPU Enabled Memory Aware Model Parallelism System for Distributed DNN Training
A. Jain, A. Awan, A. Aljuhani, J. Hashmi, Q. Anthony, H. Subramoni, D. Panda, R. Machiraju, A. Parwani
SC — IEEE/ACM International Conference for High Performance Computing, Networking, Storage, and Analysis, 2020
FALCON-X: Zero-copy MPI Derived Datatype Processing on Modern CPU and GPU Architectures
J. Hashmi, C. Chu, S. Chakraborty, M. Bayatpour, H. Subramoni, D. K. Panda
Journal of Parallel and Distributed Computing (JPDC), Vol. 144, Oct 2020, pp. 1–13
Machine-agnostic and Communication-aware Designs for MPI on Emerging Architectures
J. Hashmi, S. Xu, B. Ramesh, M. Bayatpour, H. Subramoni, D. K. Panda
IPDPS — 34th IEEE International Parallel and Distributed Processing Symposium, 2020
Slides
FALCON: Efficient Designs for Zero-copy MPI Datatype Processing on Emerging ArchitecturesBEST PAPER FINALIST
J. Hashmi, S. Chakraborty, M. Bayatpour, H. Subramoni, D. K. Panda
IPDPS — 33rd IEEE International Parallel and Distributed Processing Symposium, 2019
Slides
Designing Efficient Shared Address Space Reduction Collectives for Multi-/Many-cores
J. Hashmi, S. Chakraborty, M. Bayatpour, H. Subramoni, D. K. Panda
IPDPS — 32nd IEEE International Parallel and Distributed Processing Symposium, 2018
Slides
Design and Characterization of Shared Address Space MPI Collectives on Modern Architectures
J. Hashmi, S. Chakraborty, M. Bayatpour, H. Subramoni, D. K. Panda
CCGrid — 19th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, 2019
Slides
Cooperative Rendezvous Protocols for Improved Performance and OverlapBEST PAPER FINALIST
S. Chakraborty, M. Bayatpour, J. Hashmi, H. Subramoni, D. K. Panda
SC — IEEE/ACM International Conference for High Performance Computing, Networking, Storage, and Analysis, 2018
SALaR: Scalable and Adaptive Designs for Large Message Reduction CollectivesBEST PAPER AWARD
M. Bayatpour, S. Chakraborty, J. Hashmi, H. Subramoni, D. K. Panda
CLUSTER — IEEE International Conference on Cluster Computing, 2018
S-Caffe: Co-designing MPI Runtimes and Caffe for Scalable Deep Learning on Modern GPU Clusters
A. Awan, K. Hamidouche, J. Hashmi, D. K. Panda
PPoPP — 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2017
Kernel-assisted Communication Engine for MPI on Emerging Manycore Processors
J. Hashmi, K. Hamidouche, H. Subramoni, D. K. Panda
HiPC — 24th IEEE International Conference on High Performance Computing, Data, Analytics and Data Science, 2017
Slides

Service

// committees
  • SEP 2025Technical Program Committee, IPDPS 2026 (Architecture Track)
  • JUN 2025Technical Program Committee, HOT Interconnects 2025
  • JUN 2024Technical Program Committee, HOT Interconnects 2024
  • JUN 2023Technical Program Committee, HOT Interconnects 2023
  • JUN 2022Technical Program Committee, HOT Interconnects 2022
  • SEP 2021Technical Program Committee, IPDPS 2022 (Architecture Track)
  • AUG 2021Technical Reviewer, IEEE MICRO journal
  • JUN 2021Technical Program Committee, HOT Interconnects 2021

Recent Updates

// news
  • DEC 2021Paper selected as Best Paper Finalist at HiPC '21
  • SEP 2021Architecture-aware communication trees paper accepted to HiPC '21
  • MAR 2021ROCm-aware MPI library work accepted to ISC '21
  • MAR 2021"BluesMPI" accepted to ISC '21
  • OCT 2020"Blink" accepted to HiPC '20
  • JUN 2020"GEMS" accepted to SC '20
  • JUN 2020Started full-time as Senior Research Associate at NBCL
  • MAY 2020Successfully defended Ph.D. thesis (Slides)

Education

// 2007 — 2020
Ph.D. in Computer Science and Engineering (HPC)
The Ohio State University, Columbus, Ohio, USA — advised by Prof. D. K. Panda
2015 — 2020
M.S. in Computer Engineering
Ajou University, Suwon, South Korea — energy-efficient HPC on low-power ARM SoC clusters
2012 — 2014
B.S. in Information Technology
National University of Science and Technology (NUST), Islamabad, Pakistan
2007 — 2011