Replace backbone

[Lxy811]

❓ Questions and Help

Hello, I see some papers will be modified for backbone experiments, I want to ask I want to change to vgg16, how can the code achieve it, change the configuration file also need to change what

[Tongling-P]

Hello, have you solved the problem? How to configure the relevant files？Thank you so much for your assistance!

[Lxy811]

Hello, have you solved the problem? How to configure the relevant files？Thank you so much for your assistance!

I remember having the configuration file corresponding to the xml file at the end of the vgg, and finding the corresponding training code to retrain the pre-trained model

[Lxy811]

I don't remember it being complicated. It's the right thing to do

[Tongling-P]

Thank you for your reply. In addition to the "CONV_BODY: "R-101-FPN" # VGG-16" part, are there any other locations that need to be modified?INPUT:

MIN_SIZE_TRAIN: (600,)
 MAX_SIZE_TRAIN: 1000
 MIN_SIZE_TEST: 600
 MAX_SIZE_TEST: 1000
MODEL:
 META_ARCHITECTURE: "GeneralizedRCNN"
 WEIGHT: "catalog://ImageNetPretrained/FAIR/20171220/X-101-32x8d"
 BACKBONE:
   CONV_BODY: "R-101-FPN" # VGG-16
 RESNETS:
   BACKBONE_OUT_CHANNELS: 256
   STRIDE_IN_1X1: False
   NUM_GROUPS: 32
   WIDTH_PER_GROUP: 8
 RELATION_ON: True
 ATTRIBUTE_ON: False
 FLIP_AUG: False            # if there is any left-right relation, FLIP AUG should be false

[Tongling-P]

When I make just one change I get the following error：
Traceback (most recent call last): File "tools/HTCL_main.py", line 153, in <module> main() File "tools/HTCL_main.py", line 97, in main htcl_train_sgg_net(cfg, args.local_rank, args.distributed, logger) File "/home/HTCL/maskrcnn_benchmark/HTCL/htcl_train_sgg_net.py", line 114, in htcl_train_sgg_net checkpointer.load(cfg.MODEL.PRETRAINED_DETECTOR_CKPT, with_optim=False, load_mapping=load_mapping) File "/home/HTCL/maskrcnn_benchmark/utils/checkpoint.py", line 65, in load self._load_model(checkpoint, load_mapping) File "/home/HTCL/maskrcnn_benchmark/utils/checkpoint.py", line 107, in _load_model load_state_dict(self.model, checkpoint.pop("model"), load_mapping) File "/home/HTCL/maskrcnn_benchmark/utils/model_serialization.py", line 94, in load_state_dict model.load_state_dict(model_state_dict) File "/root/miniconda3/envs/HTCL/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1483, in load_state_dict self.__class__.__name__, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for GeneralizedRCNN: size mismatch for rpn.anchor_generator.cell_anchors.0: copying a param with shape torch.Size([4, 4]) from checkpoint, the shape in current model is torch.Size([20, 4]). size mismatch for rpn.head.conv.weight: copying a param with shape torch.Size([256, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([512, 512, 3, 3]). size mismatch for rpn.head.conv.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]). size mismatch for rpn.head.cls_logits.weight: copying a param with shape torch.Size([4, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([20, 512, 1, 1]). size mismatch for rpn.head.cls_logits.bias: copying a param with shape torch.Size([4]) from checkpoint, the shape in current model is torch.Size([20]). size mismatch for rpn.head.bbox_pred.weight: copying a param with shape torch.Size([16, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([80, 512, 1, 1]). size mismatch for rpn.head.bbox_pred.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([80]). size mismatch for roi_heads.box.feature_extractor.fc6.weight: copying a param with shape torch.Size([4096, 12544]) from checkpoint, the shape in current model is torch.Size([2048, 25088]). size mismatch for roi_heads.box.feature_extractor.fc6.bias: copying a param with shape torch.Size([4096]) from checkpoint, the shape in current model is torch.Size([2048]). size mismatch for roi_heads.box.feature_extractor.fc7.weight: copying a param with shape torch.Size([4096, 4096]) from checkpoint, the shape in current model is torch.Size([2048, 2048]). size mismatch for roi_heads.box.feature_extractor.fc7.bias: copying a param with shape torch.Size([4096]) from checkpoint, the shape in current model is torch.Size([2048]). size mismatch for roi_heads.box.predictor.cls_score.weight: copying a param with shape torch.Size([151, 4096]) from checkpoint, the shape in current model is torch.Size([151, 2048]). size mismatch for roi_heads.box.predictor.bbox_pred.weight: copying a param with shape torch.Size([604, 4096]) from checkpoint, the shape in current model is torch.Size([604, 2048]). size mismatch for roi_heads.relation.union_feature_extractor.feature_extractor.fc6.weight: copying a param with shape torch.Size([4096, 12544]) from checkpoint, the shape in current model is torch.Size([2048, 25088]). size mismatch for roi_heads.relation.union_feature_extractor.feature_extractor.fc6.bias: copying a param with shape torch.Size([4096]) from checkpoint, the shape in current model is torch.Size([2048]). size mismatch for roi_heads.relation.union_feature_extractor.feature_extractor.fc7.weight: copying a param with shape torch.Size([4096, 4096]) from checkpoint, the shape in current model is torch.Size([2048, 2048]). size mismatch for roi_heads.relation.union_feature_extractor.feature_extractor.fc7.bias: copying a param with shape torch.Size([4096]) from checkpoint, the shape in current model is torch.Size([2048]). size mismatch for roi_heads.relation.box_feature_extractor.fc6.weight: copying a param with shape torch.Size([4096, 12544]) from checkpoint, the shape in current model is torch.Size([2048, 25088]). size mismatch for roi_heads.relation.box_feature_extractor.fc6.bias: copying a param with shape torch.Size([4096]) from checkpoint, the shape in current model is torch.Size([2048]). size mismatch for roi_heads.relation.box_feature_extractor.fc7.weight: copying a param with shape torch.Size([4096, 4096]) from checkpoint, the shape in current model is torch.Size([2048, 2048]). size mismatch for roi_heads.relation.box_feature_extractor.fc7.bias: copying a param with shape torch.Size([4096]) from checkpoint, the shape in current model is torch.Size([2048]).

[Tongling-P]

Sorry to bother you, I am looking forward to your reply, your reply will be of great help to me!

[Lxy811]

Sorry to bother you, I am looking forward to your reply, your reply will be of great help to me!

Hello, I haven't looked at this work for a long time and have forgotten a lot of work, but recently I may keep looking at the code and wait for a few days to communicate. Then the problem you have here seems to be a dimension mismatch, and you need to retrain a new checkpoint.

[Tongling-P]

Thank you for your reply, the problem has been solved! In addition, do you know how to change the dataset for training?

[Lxy811]

Thank you for your reply, the problem has been solved! In addition, do you know how to change the dataset for training?

This website seems to have steps for preprocessing the data set, and the annotation tool should be able to find it by searching genome on github, similar to the labelImg tool. I don't remember the details now.

[Tongling-P]

Thank you for your answer! Do you remember how to replace VG with OpenImage dataset?

[Lxy811]

Thank you for your answer! Do you remember how to replace VG with OpenImage dataset?

I don't know. I hope you can solve it. Recently, I wondered how to make AG dynamic dataset. If you have a good method, I hope to discuss it together