ShowMeAI资讯日报 2022-06-16 期，【腾讯】分布式 embedding 训练和推理框架、【蚂蚁】时序数据库、【搜狗】C++服务器引擎、正则表达式实例清单、MongoDB全方位知识图谱、8篇前沿论文…点击获取全部资讯

ShowMeAI日报系列全新升级！覆盖AI人工智能 工具&框架 | 项目&代码 | 博文&分享 | 数据&资源 | 研究&论文 等方向。点击查看 历史文章列表，在公众号内订阅话题 #ShowMeAI资讯日报，可接收每日最新推送。点击 专题合辑&电子月刊 快速浏览各专题全集。点击 这里 回复关键字 日报 免费获取AI电子月刊与资料包。

embedx 是基于 c++ 开发的大规模 embedding 训练和推理系统，累计支持公司 12 个业务、 30 多个团队使用、上线百余次。

GitHub:

tags:[数据库，时序数据库，Rust]

GitHub:

它支撑搜狗几乎所有后端C++在线服务，包括所有搜索服务，云输入法，在线广告等，每日处理超百亿请求。这是一个设计轻盈优雅的企业级程序引擎，可以满足大多数C++后端开发需求。

GitHub:

'fccf: A command-line tool that quickly searches through C/C++ source code in a directory based on a search string and prints relevant code snippets that match the query.' by Pranav

GitHub:

'Open3DSOT - Open source library for Single Object Tracking in point clouds.' by Kangel Zenn

GitHub:

tags:[电子书，web]

谷歌公司高性能团队核心成员的权威之作《High Performance Browser Networking》，涵盖Web 开发者技术体系中应该掌握的所有网络及性能优化知识。全书以性能优化为主线，从TCP、UDP 和TLS 协议讲起，解释了如何针对这几种协议和基础设施来优化应用。然后深入探讨了无线和移动网络的工作机制。最后，揭示了HTTP 协议的底层细节，同时详细介绍了HTTP 2.0、 XHR、SSE、WebSocket、WebRTC 和DataChannel 等现代浏览器新增的具有革命性的新能力。

GitHub:

Link:

对 MongoDB 比较全面深入的博文介绍。

Link

本项目基于数学推导和产生原理重新描述了书中的概念，并用Python (numpy 库为主) 复现了书本内容 ( 源码级代码实现。《深度学习》涉及到的每一个概念，都会去给它详细的描述、原理层面的推导，以及用代码的实现。代码实现不会调用 Tensorflow、PyTorch、MXNet 等任何深度学习框架，甚至包括 sklearn (pdf 里用到 sklearn 的部分都是用来验证代码无误)，一切代码都是从原理层面实现 (Python 的基础库 NumPy)，并有详细注释，与代码区上方的原理描述区一致，你可以结合原理和代码一起理解。

GitHub:

可以点击 这里 回复关键字 日报，免费获取整理好的6月论文合辑。

论文标题：Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

论文时间：9 Jun 2022

所属领域：推理推断，自然语言处理

对应任务：常识推理，文本理解

论文地址：

代码实现：

论文作者：Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb等

论文简介：BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language modelshttps://zhuanlan.zhihu.com/p/BIG bench专注于被认为超出当前语言模型能力的任务。

论文摘要：Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabilities are as yet poorly characterized. In order to inform future research, prepare for disruptive new model capabilities, and ameliorate socially harmful effects, it is vital that we understand the present and near-future capabilities and limitations of language models. To address this challenge, we introduce the Beyond the Imitation Game benchmark (BIG-bench). BIG-bench currently consists of 204 tasks, contributed by 442 authors across 132 institutions. Task topics are diverse, drawing problems from linguistics, childhood development, math, common-sense reasoning, biology, physics, social bias, software development, and beyond. BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models. We evaluate the behavior of OpenAI's GPT models, Google-internal dense transformer architectures, and Switch-style sparse transformers on BIG-bench, across model sizes spanning millions to hundreds of billions of parameters. In addition, a team of human expert raters performed all tasks in order to provide a strong baseline. Findings include: model performance and calibration both improve with scale, but are poor in absolute terms (and when compared with rater performance); performance is remarkably similar across model classes, though with benefits from sparsity; tasks that improve gradually and predictably commonly involve a large knowledge or memorization component, whereas tasks that exhibit "breakthrough" behavior at a critical scale often involve multiple steps or components, or brittle metrics; social bias typically increases with scale in settings with ambiguous context, but this can be improved with prompting.

随着规模的不断扩大，语言模型显示出定量改进和新的定性能力。尽管这些新功能具有潜在的变革性影响，但其特征尚不明确。为了为未来的研究提供信息，为破坏性的新模型能力做好准备，并改善对社会有害的影响，我们必须了解语言模型目前和近期的能力和局限性。为了应对这一挑战，我们引入了超越模仿游戏基准（BIG-bench）。BIG bench目前由204项任务组成，由132个机构的442名作者贡献。任务主题多种多样，涉及语言学、儿童发展、数学、常识推理、生物学、物理学、社会偏见、软件开发等领域的问题。BIG bench专注于被认为超出当前语言模型能力的任务。我们评估了OpenAI的GPT模型、Google内部密集Transformer架构和大平台上的开关式稀疏Transformer的行为，模型大小跨越数百万到数千亿个参数。此外，一组人类专家评分员完成了所有任务，以提供一个强大的基线。研究结果包括：模型性能和校准都随规模的增加而改善，但在绝对值方面较差（与评分员性能相比）；模型类的性能非常相似，尽管稀疏性带来了好处；逐渐提高和可预测的任务通常涉及大量的知识或记忆成分，而在关键尺度上表现出“突破”行为的任务通常涉及多个步骤或组件，或脆弱的指标；在背景不明确的环境中，社会偏见通常会随着规模的增加而增加，但这可以通过prompt得到改善。

论文标题：Dual-Distribution Discrepancy for Anomaly Detection in Chest X-Rays

论文时间：8 Jun 2022

所属领域：Methodology

对应任务：Anomaly Detection，异常检测

论文地址：

代码实现：

论文作者：Yu Cai, Hao Chen, Xin Yang, Yu Zhou, Kwang-Ting Cheng

论文简介：During training, module A takes both known normal and unlabeled images as inputs, capturing anomalous features from unlabeled images in some way, while module B models the distribution of only known normal images. / 在训练期间，模块 A 将已知正常和未标记图像作为输入，以某种方式从未标记图像中捕获异常特征，而模块 B 仅对已知正常图像的分布进行建模。

论文摘要：Chest X-ray (CXR) is the most typical radiological exam for diagnosis of various diseases. Due to the expensive and time-consuming annotations, detecting anomalies in CXRs in an unsupervised fashion is very promising. However, almost all of the existing methods consider anomaly detection as a One-Class Classification (OCC) problem. They model the distribution of only known normal images during training and identify the samples not conforming to normal profile as anomalies in the testing phase. A large number of unlabeled images containing anomalies are thus ignored in the training phase, although they are easy to obtain in clinical practice. In this paper, we propose a novel strategy, Dual-distribution Discrepancy for Anomaly Detection (DDAD), utilizing both known normal images and unlabeled images. The proposed method consists of two modules, denoted as A and B. During training, module A takes both known normal and unlabeled images as inputs, capturing anomalous features from unlabeled images in some way, while module B models the distribution of only known normal images. Subsequently, the inter-discrepancy between modules A and B, and intra-discrepancy inside module B are designed as anomaly scores to indicate anomalies. Experiments on three CXR datasets demonstrate that the proposed DDAD achieves consistent, significant gains and outperforms state-of-the-art methods. Code is available at

胸部 X 线（CXR）是诊断各种疾病的最典型的放射检查。由于注释昂贵且耗时，以无监督的方式检测 CXR 中的异常是非常有前途的。然而，几乎所有现有的方法都将异常检测视为一类分类（OCC）问题。他们在训练期间仅对已知正常图像的分布进行建模，并在测试阶段将不符合正常轮廓的样本识别为异常。因此，在训练阶段会忽略大量包含异常的未标记图像，尽管它们在临床实践中很容易获得。在本文中，我们提出了一种新颖的策略，即异常检测的双分布差异 (DDAD)，同时利用已知的正常图像和未标记的图像。所提出的方法由两个模块组成，分别表示为 A 和 B。在训练期间，模块 A 将已知的正常和未标记图像作为输入，以某种方式从未标记图像中捕获异常特征，而模块 B 仅对已知正常图像的分布进行建模.随后，将模块 A 和 B 之间的相互差异以及模块 B 内部的内部差异设计为异常分数以指示异常。对三个 CXR 数据集的实验表明，所提出的 DDAD 实现了一致、显着的收益，并且优于最先进的方法。代码在

论文标题：Sparse Fusion Mixture-of-Experts are Domain Generalizable Learners

论文时间：8 Jun 2022

所属领域：计算机视觉

对应任务：Domain Generalization，Object Recognition，领域泛化，物体识别

论文地址：

代码实现：

论

人工智能与深度学习技术16大回顾，承包你一年的知识点