Deep Reinforcement Learning for Solving Problems in Systems and Chip Design
, Google Brain
, Google Brain
Many core problems in systems and hardware design are combinatorial optimization or decision-making tasks with state and action spaces that are orders of magnitude larger than those of standard AI benchmarks in robotics and games. We'll describe some of our latest learning-based approaches to tackling such large-scale optimization problems. We'll discuss our work on a new domain-transferable reinforcement learning method for optimizing chip placement, a long pole in hardware design. Our approach learns from experience and improves over time, resulting in more optimized placements on unseen chip blocks as the RL agent is exposed to more data. Our objective is to minimize power, performance, and area, and we show that, in under six hours, our method can generate placements that are superhuman or comparable on modern accelerator chips, whereas existing baselines require human experts in the loop and can take several weeks.