CURRENTLY AT NVIDIA

Jahanzeb Hashmi

Senior Architect

Resume CV Google Scholar

About

// summary
Jahanzeb Hashmi

I am a Senior Architect / Technical Lead at NVIDIA, where I drive technical strategy for next-generation GPU and datacenter systems, collaborating with cross-functional teams spanning Hardware, Software, and Product to define architecture direction and evaluate design trade-offs from chip-level to rack-scale NVLink platforms. My work has shaped multiple NVIDIA GPU generations—from Hopper through Rubin and beyond—through architectural feature exploration, hardware/software co-design, workload modeling, and competitive analysis, with recommendations presented to executive leadership, including the CEO.

Before joining NVIDIA, I was a Senior Research Associate at The Ohio State University, working in the Network-Based Computing Laboratory on MVAPICH2, a high-performance MPI library used across supercomputing centers worldwide. My focus was GPU-aware MPI for RDMA networks on AMD and NVIDIA hardware, hierarchical collective algorithms for scalable scientific and deep learning workloads, zero-copy data movement for sparse layouts, and performance engineering of parallel applications on multi-petaflop systems.

Experience

// 2015 — present
MAR 2021 — PRESENT

Senior Architect / Technical Lead

NVIDIA Corporation, USA
  • Leading cross-org path-finding efforts to analyze design trad-offs of building scale-up and scale-out systems. My work provides insights into NVIDIA's product positioning and is directly reported to E-staff and the CEO of the company.
  • Leading the architectural feature exploration and value proposition of next-generation NVIDIA GPU architecture for HPC, Industrial, and AI applications. I have contributed to several design PORs and provided quantitative data to get them approved across several generations of NVIDIA products starting from Hopper to Rubin and several upcoming products.
  • Identify application scaling challenges and co-design applications across HW/SW design space to achieve maximum performance, energy efficiency, and TCO on NVIDIA platforms.
  • Model business critical workloads to establish performance targets for NVIDIA's upcoming architectures to serve as north star for system bringup teams.
  • Lead the HPC competitive analysis working group, projecting application and system performance — from kernel/chip-level analysis to full-scale datacenter architecture — on competitive GPU/accelerator products
JUN 2020 — MAR 2021

Senior Research Associate

The Ohio State University, USA
  • Designed a high-performance MPI library for next-gen HPC/cloud systems spanning multi-core CPUs and GPUs (NVIDIA, AMD).
  • Led design of a generalized hierarchical MPI collective communications framework optimized for modern CPU+GPU node architectures.
  • Led MVAPICH2-GDR, a GPU-aware MPI library for NVIDIA/AMD multi-GPU systems, from architecture through production deployment.
  • Mentored Ph.D. and M.S. students on communication runtime design for CPUs, GPUs, and high-speed interconnects.
AUG 2015 — MAY 2020

Graduate Research Associate

The Ohio State University, USA
  • Designed an adaptive, topology-aware algorithm mapping MPI processes to hardware cores based on communication patterns of AI/HPC workloads.
  • Collaborated on efficient parallelization of large-scale distributed DNN training (data- and model-parallel) across CPU/GPU systems.
  • Designed a zero-copy XPMEM-based inter-process communication layer for manycore/multi-GPU architectures.
  • Designed a data-layout caching algorithm mitigating MPI derived-datatype translation costs — Best Paper Award nominee, IPDPS '19.

Skills

// tools & stacks

GPU & Accelerator Systems

CUDA / HIPNCCL GPU-aware MPI/SHMEMNVIDIA Nsight Suite Multi-GPU/Multi-node Profiling

Performance Engineering

Roofline ModelingFLOPs / Bandwidth Analysis Interconnect Throughput

System Design

HW/SW Co-designEnd-to-end Perf/Power/TCO Optimization

Languages

CC++PythonJava

Select Publications

// peer-reviewed

For the complete list, see my Google Scholar page.

Towards Architecture-aware Hierarchical Communication Trees on Modern HPC SystemsBEST PAPER FINALIST
B. Ramesh, J. Hashmi, S. Xu, A. Shafi, M. Ghazimirsaeed, M. Bayatpour, H. Subramoni, D. K. Panda
HiPC — 28th IEEE International Conference on High Performance Computing, Data, Analytics and Data Science, 2021
GEMS: GPU Enabled Memory Aware Model Parallelism System for Distributed DNN Training
A. Jain, A. Awan, A. Aljuhani, J. Hashmi, Q. Anthony, H. Subramoni, D. Panda, R. Machiraju, A. Parwani
SC — IEEE/ACM International Conference for High Performance Computing, Networking, Storage, and Analysis, 2020
FALCON-X: Zero-copy MPI Derived Datatype Processing on Modern CPU and GPU Architectures
J. Hashmi, C. Chu, S. Chakraborty, M. Bayatpour, H. Subramoni, D. K. Panda
Journal of Parallel and Distributed Computing (JPDC), Vol. 144, Oct 2020, pp. 1–13
Machine-agnostic and Communication-aware Designs for MPI on Emerging Architectures
J. Hashmi, S. Xu, B. Ramesh, M. Bayatpour, H. Subramoni, D. K. Panda
IPDPS — 34th IEEE International Parallel and Distributed Processing Symposium, 2020
Slides
FALCON: Efficient Designs for Zero-copy MPI Datatype Processing on Emerging ArchitecturesBEST PAPER FINALIST
J. Hashmi, S. Chakraborty, M. Bayatpour, H. Subramoni, D. K. Panda
IPDPS — 33rd IEEE International Parallel and Distributed Processing Symposium, 2019
Slides
Designing Efficient Shared Address Space Reduction Collectives for Multi-/Many-cores
J. Hashmi, S. Chakraborty, M. Bayatpour, H. Subramoni, D. K. Panda
IPDPS — 32nd IEEE International Parallel and Distributed Processing Symposium, 2018
Slides
Design and Characterization of Shared Address Space MPI Collectives on Modern Architectures
J. Hashmi, S. Chakraborty, M. Bayatpour, H. Subramoni, D. K. Panda
CCGrid — 19th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, 2019
Slides
Cooperative Rendezvous Protocols for Improved Performance and OverlapBEST PAPER FINALIST
S. Chakraborty, M. Bayatpour, J. Hashmi, H. Subramoni, D. K. Panda
SC — IEEE/ACM International Conference for High Performance Computing, Networking, Storage, and Analysis, 2018
SALaR: Scalable and Adaptive Designs for Large Message Reduction CollectivesBEST PAPER AWARD
M. Bayatpour, S. Chakraborty, J. Hashmi, H. Subramoni, D. K. Panda
CLUSTER — IEEE International Conference on Cluster Computing, 2018
S-Caffe: Co-designing MPI Runtimes and Caffe for Scalable Deep Learning on Modern GPU Clusters
A. Awan, K. Hamidouche, J. Hashmi, D. K. Panda
PPoPP — 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2017
Kernel-assisted Communication Engine for MPI on Emerging Manycore Processors
J. Hashmi, K. Hamidouche, H. Subramoni, D. K. Panda
HiPC — 24th IEEE International Conference on High Performance Computing, Data, Analytics and Data Science, 2017
Slides

Service

// committees
  • SEP 2025Technical Program Committee, IPDPS 2026 (Architecture Track)
  • JUN 2025Technical Program Committee, HOT Interconnects 2025
  • JUN 2024Technical Program Committee, HOT Interconnects 2024
  • JUN 2023Technical Program Committee, HOT Interconnects 2023
  • JUN 2022Technical Program Committee, HOT Interconnects 2022
  • SEP 2021Technical Program Committee, IPDPS 2022 (Architecture Track)
  • AUG 2021Technical Reviewer, IEEE MICRO journal
  • JUN 2021Technical Program Committee, HOT Interconnects 2021

Recent Updates

// news
  • DEC 2021Paper selected as Best Paper Finalist at HiPC '21
  • SEP 2021Architecture-aware communication trees paper accepted to HiPC '21
  • MAR 2021ROCm-aware MPI library work accepted to ISC '21
  • MAR 2021"BluesMPI" accepted to ISC '21
  • OCT 2020"Blink" accepted to HiPC '20
  • JUN 2020"GEMS" accepted to SC '20
  • JUN 2020Started full-time as Senior Research Associate at NBCL
  • MAY 2020Successfully defended Ph.D. thesis (Slides)

Education

// 2007 — 2020
Ph.D. in Computer Science and Engineering (HPC)
The Ohio State University, Columbus, Ohio, USA — advised by Prof. D. K. Panda
2015 — 2020
M.S. in Computer Engineering
Ajou University, Suwon, South Korea — energy-efficient HPC on low-power ARM SoC clusters
2012 — 2014
B.S. in Information Technology
National University of Science and Technology (NUST), Islamabad, Pakistan
2007 — 2011