AI Ants

迈向通用人工智能之路

0%

1、Flops、MACs、Params计算

https://github.com/he-y/soft-filter-pruning/blob/master/utils/cifar_resnet_flop.py

https://github.com/warmspringwinds/pytorch-segmentation-detection/blob/master/pytorch_segmentation_detection/utils/flops_benchmark.py

https://github.com/Lyken17/pytorch-OpCounter

crnn中:

1
2
3
4
5
6
7
8
9
10
统计模型参数量等信息
def model_info(model): # Plots a line-by-line description of a PyTorch model
n_p = sum(x.numel() for x in model.parameters()) # number parameters
n_g = sum(x.numel() for x in model.parameters() if x.requires_grad) # number gradients
print('\n%5s %50s %9s %12s %20s %12s %12s' % ('layer', 'name', 'gradient', 'parameters', 'shape', 'mu', 'sigma'))
for i, (name, p) in enumerate(model.named_parameters()):
name = name.replace('module_list.', '')
print('%5g %50s %9s %12g %20s %12.3g %12.3g' % (
i, name, p.requires_grad, p.numel(), list(p.shape), p.mean(), p.std()))
print('Model Summary: %g layers, %g parameters, %g gradients\n' % (i + 1, n_p, n_g))

2、模型初始化

1
2
3
4
5
6
7
8
9
10
11
12
def weights_init(model):
# Official init from torch repo.
for m in model.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_normal_(m.weight)
elif isinstance(m, nn.BatchNorm2d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0)
elif isinstance(m, nn.Linear):
nn.init.constant_(m.bias, 0)
encoder = crnn.CNN(opt.imgH, nc, opt.nh, cfg, mode='2D',dim_in=512)
encoder.apply(weights_init)

3、导入模型(单机单卡到分布式)

4、自定义学习率(如warmup)和优化器

网络变种

1、PyTorch-GAN

https://github.com/eriklindernoren/PyTorch-GAN

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Table of Contents
Installation
Implementations
Auxiliary Classifier GAN
Adversarial Autoencoder
BEGAN
BicycleGAN
Boundary-Seeking GAN
Cluster GAN
Conditional GAN
Context-Conditional GAN
Context Encoder
Coupled GAN
CycleGAN
Deep Convolutional GAN
DiscoGAN
DRAGAN
DualGAN
Energy-Based GAN
Enhanced Super-Resolution GAN
GAN
InfoGAN
Least Squares GAN
MUNIT
Pix2Pix
PixelDA
Relativistic GAN
Semi-Supervised GAN
Softmax GAN
StarGAN
Super-Resolution GAN
UNIT
Wasserstein GAN
Wasserstein GAN GP
Wasserstein GAN DIV

应用

风格迁移

1、pystiche

https://github.com/pmeier/pystiche

Framework for Neural Style Transfer (NST) built upon PyTorch

2、GMA_CI_2019_ssim_content_loss

https://github.com/pmeier/GMA_CI_2019_ssim_content_loss

Content representation for Neural Style Transfer Algorithms based on Structural Similarity

Anchor的生成和匹配(正负样本的选取)

NMS

Loss

样本(类别)不均衡(长尾分布)

特征对齐

1、ROI pooling

小目标检测优化

细长目标检测优化

速度和精度的trade off

蒸馏

1、Distilling Object Detectors with Fine-grained Feature Imitation(CVPR19)

https://github.com/twangnh/Distilling-Object-Detectors

剪枝

1、soft-filter-pruning(FPGM)

https://github.com/he-y/filter-pruning-geometric-median?utm_source=catalyzex.com

Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration(CVPR19 Oral)

量化

1、EasyQuant: Post-training Quantization via Scale Optimization

https://github.com/deepglint/EasyQuant

EasyQuant(EQ) is an efficient and simple post-training quantization method via effectively optimizing the scales of weights and activations

2、dnn-gating(PACT)

https://github.com/cornell-zhang/dnn-gating?utm_source=catalyzex.com

PACT: PARAMETERIZED CLIPPING ACTIVATION FOR QUANTIZED NEURAL NETWORKS

3、scale-adjusted-training

https://github.com/jakc4103/scale-adjusted-training?utm_source=catalyzex.com

Towards Efficient Training for Neural Network Quantization