Sitemap
A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.
Pages
Posts
NCCL源码阅读笔记
Published:
最近简单读了一下NVIDIA Collective Communications Library (NCCL, pronounced “Nickel”) 的源代码。现将初步的一些收获总结一下。 Read more
What drives performance improvement of Deep Learning Models in recent years
Published:
这一周主要是看论文，浏览了今年MLSys，ASPLOS以及去年SOSP和前年OSDI上与系统和机器学习相关的文章，重点阅读了一些与深度学习的性能提升相关的文章。受到之前包老师推荐阅读的文章（There’s plenty of room at the Top）的影响，我把这些文章分为三类，分别是Algorithm、Software和Hardware Architecture。由于我关注的这些会议都是偏系统的，所以Software层面的文章比较多。 Read more
GPU Performance: Black-box and White-box Perspectives
Published:
最近调研了一些GPU性能模型相关的文章。这些模型关注通用场景下GPU kernel的运算性能，往往将GPU kernel看做一个黑盒子（而事实也确实如此），通过benchmark以及profiling tools给出的一些信息，来对kernel的运行时间进行预测。同时，在另一方面，为了能深入理解程序员对CUDA kernel的优化过程，我也阅读了NVIDIA提供的CUDA Best Practices Guide[1]。直接从程序员编写CUDA代码的角度来理解CUDA kernel可能存在的一些特性。现在将这两部分内容做一些整理和总结，从两个角度来对GPU kernel的性能进行分析。 Read more
Performance Model of Convolution on GPU
Published:
背景
GPU Introduction
- GPU架构 NVIDIA GPU包含多个Stream Multiprocessor（SM）。每个SM的结构如上图所示，其中包含多个Stream Processor（SP）以及数个Special Function Units (SFU)，SP是GPU的基本算术指令执行单元，SFU则用于执行一些复杂的操作，例如sin，cos等。除此之外，SM还包含一个几十KB大小的Shared-Memory、一个Instruction Cache和Data Cache，以及一个Multithreading issuing unit用于指令的调度。
分布式系统和机器学习-文献阅读报告
Published:
摘要
我选择的20篇文章主要关注分布式系统，以及分布式系统在机器学习中的应用。分布式系统是计算机科学历史悠久的一个研究方向，设计一个分布式系统需要考虑许多方面的因素，例如进程的交互与通信协议、故障处理及容错机制、数据复制及一致性等。经过学界多年的研究，针对这些问题，已经有了比较好的解决方案。而在当下，机器学习正迅速发展，越来越多的数据以及规模越来越大的模型，都使得在分布式环境中训练机器学习模型成为一大主流趋势。如何针对机器学习应用设计分布式系统，从而加速机器学习应用的训练过程，成为现在研究的热潮。 Read more分布式深度学习-Survey
Published:
Intro
由于神经网络的规模越来越大，单个计算设备往往难以在短时间内完成对整个网络的训练，因此出现了大量的分布式神经网络训练方案，利用多个节点的计算能力加速训练过程。 而计算设备也在不断更新，可以是CPU、GPU以及TPU等更多类型的硬件。本文调研了分布式深度学习领域相关的最新文献，简要介绍该领域目前关注的主要问题和已有的解决方案。 Read moreportfolio
Portfolio item number 1
Short description of portfolio item number 1
Read more
Portfolio item number 2
Short description of portfolio item number 2 Read more
publications
Paper Title Number 1
Published in Journal 1, 2009
This paper is about the number 1. The number 2 is left for future work. Read more
Recommended citation: Your Name, You. (2009). "Paper Title Number 1." Journal 1. 1(1). http://academicpages.github.io/files/paper1.pdf
Paper Title Number 2
Published in Journal 1, 2010
This paper is about the number 2. The number 3 is left for future work. Read more
Recommended citation: Your Name, You. (2010). "Paper Title Number 2." Journal 1. 1(2). http://academicpages.github.io/files/paper2.pdf
Paper Title Number 3
Published in Journal 1, 2015
This paper is about the number 3. The number 4 is left for future work. Read more
Recommended citation: Your Name, You. (2015). "Paper Title Number 3." Journal 1. 1(3). http://academicpages.github.io/files/paper3.pdf
Paper Title Number 4
Published in GitHub Journal of Bugs, 2024
This paper is about fixing template issue #693. Read more
Recommended citation: Your Name, You. (2024). "Paper Title Number 3." GitHub Journal of Bugs. 1(3). http://academicpages.github.io/files/paper3.pdf
talks
Talk 1 on Relevant Topic in Your Field
Published:
This is a description of your talk, which is a markdown files that can be all markdown-ified like any other post. Yay markdown! Read more
Conference Proceeding talk 3 on Relevant Topic in Your Field
Published:
This is a description of your conference proceedings talk, note the different field in type. You can put anything in this field. Read more
teaching
Teaching experience 1
Undergraduate course, University 1, Department, 2014
This is a description of a teaching experience. You can use markdown like any other post. Read more
Teaching experience 2
Workshop, University 1, Department, 2015
This is a description of a teaching experience. You can use markdown like any other post. Read more