Finetuning on VisLoc

[Thiagor2002]

I want to know how do you finetune on VisLoc dataset using your pretrained vit16/convnext model of GTA-UAV dataset. Here are two sub-issues:

1. How do you organize your VisLoc dataset? Do you use all 11 subsets in UAV_VisLoc? In UAV_VisLoc, subset '07' is very small and subset '09' is large, satellite image in '09' has been split into 4 parts. have you consider processing sat_09 in scripts/prepare_dataset/visloc.py?
2. How do you finetune your vit16/convnext model, just use code train_visloc.py ? I have use train_gta.py to pretrain on GTA-UAV and achieve comparable scores, about 85-87 at Recall@1 (GTA-UAV-LR dataset, same_area ). But when I use the best convnext model to finetune on VisLoc by train_visloc.py, I only achieve 40-45 at Recall@1 in same_area scenario, far less than the 80% at Recall@1 released on your paper.

Could you please introduce me about the issues? Thank you.

[Yux1angJi]

1. Yes, we exclude some regions entirely. In the experiments reported in the paper, we use 7 regions [1, 2, 3, 4, 5, 8, 11].
2. This result doesn’t look quite right. Did you try to train on UAV-VisLoc without pretraining on GTA-UAV?

[Thiagor2002]

1. Yes, we exclude some regions entirely. In the experiments reported in the paper, we use 7 regions [1, 2, 3, 4, 5, 8, 11].
2. This result doesn’t look quite right. Did you try to train on UAV-VisLoc without pretraining on GTA-UAV?

Thanks a lot. For finetuning, firstly, I use GTA-UAV to pretrain on GTA-UAV-LR with train_gta.py. Then I use the best model to train on UAV-VisLoc with train_visloc.py, which i use region [1, 2, 3, 4, 5, 6, 8, 10, 11]. I guess it may be various for VisLoc regions selected.

[Thiagor2002]

1. Yes, we exclude some regions entirely. In the experiments reported in the paper, we use 7 regions [1, 2, 3, 4, 5, 8, 11].
2. This result doesn’t look quite right. Did you try to train on UAV-VisLoc without pretraining on GTA-UAV?

I wonder whether you have adapt other methods to improve finetuning performance such as PEFT (which you have show in your code models/vit_adapter.py by SelaVPR method), or just re-train by the script train_visloc.py and set frozen layers and frozen blocks as default(would you have try other finetuning params)?
what do train_gta_mm.py and train_gta_rgbd.py doing?
Besides, I want to know about code implementation on the Mutually Exclusive Sampling process introduced in your paper. Do you have integrate it in dataset/visloc.py or trainer?
Appreciate for your answering.

[Yux1angJi]

The reported results should be just by re-training the model on UAV-VisLoc data.
train_gta_mm.py and train_gta_rgbd.py are for training with additional modalities.
Yes, we already integrated it in both dataset/gta.py and dataset/visloc.py.

[Thiagor2002]

I find in your finetuning code, in line 77-79 of model.py file you have not given function freeze_layers， but in finetuing code train_visloc.py you have call the function.

Besides, I would like to know what the code train_gta_mm.py and train_gta_rgbd.py are used for?
Thanks for your help!

[hitersyw]

I find in your finetuning code, in line 77-79 of model.py file you have not given function freeze_layers， but in finetuing code train_visloc.py you have call the function.

Besides, I would like to know what the code train_gta_mm.py and train_gta_rgbd.py are used for? Thanks for your help!

have you solved the finetuning problem on VisLoc?