Scenic: A JAX Library for Computer Vision Research and Beyond
Mostafa Dehghani Google Brain Alexey Gritsenko Google Brain Anurag Arnab Google Research Matthias Minderer Google Brain Yi Tay Google Research
Abstract
Scenic is an open-source111https://github.com/google-research/scenic JAX library with a focus on Transformer-based models for computer vision research and beyond. The goal of this toolkit is to facilitate rapid experimentation, prototyping, and research of new vision architectures and models. Scenic supports a diverse range of vision tasks (e.g., classification, segmentation, detection) and facilitates working on multi-modal problems, along with GPU/TPU support for multi-host, multi-device large-scale training. Scenic also offers optimized implementations of state-of-the-art research models spanning a wide range of modalities. Scenic has been successfully used for numerous projects and published papers and continues serving as the library of choice for quick prototyping and publication of new research ideas.
中文速览
视觉领域的研究者在快速验证新架构想法时，往往缺少一套既灵活又高效的统一工具链。谷歌研究团队为此开源了 Scenic——一个基于 JAX 框架、以 Transformer 为核心的计算机视觉研究库，通过"库级代码"与"项目级代码"两层结构，让研究者既能直接复用数据管道、模型、训练循环等通用组件，也能随时分叉（fork）任意部分进行深度定制，而无需与臃肿的抽象层博弈。Scenic 已内置 ViT、DETR、MLP-Mixer、ResNet、U-Net 等主流模型，并在图像、视频、音频、文本等多种模态任务上完成验证，支持多机多卡的大规模 GPU/TPU 训练。它已被用于 ViViT、OmniNet、TokenLearner 等多项已发表研究，对整个视觉研究社区而言是一个能够显著压缩"想法到实验"周期的实用基础设施。
原文 arXiv:2110.11403;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2110.11403v1