FishServer/FishMeasure/weight_estimator

Weight estimator (PointNet++)

This module is for predicting fish weight from a 3D point cloud using a PointNet++-based network.

Data layout

• Point clouds: /home/ubuntu/data/fish/2025-11-19-output/
  • Structure: xxxx/cloud/xx.ply
  • Example: /home/ubuntu/data/fish/2025-11-19-output/HD1080_SN43186771_21-31-13/cloud/cloud_0708_frame_000708.ply
• Labels: /home/ubuntu/projects/FishMeasure/measure/data/label.csv
  • The 2nd column is the sample id / folder name (e.g. HD1080_SN43186771_21-31-13)
  • A later column contains the ground-truth weight (grams) used as the regression target.

Dataset mapping (how a .ply gets a label)

1. Read label.csv.
2. For each row, take column 2 as sample_id (folder name).
3. Collect all point clouds under:

/home/ubuntu/data/fish/2025-11-19-output/{sample_id}/cloud/*.ply

4. Assign the same ground-truth weight for that sample_id to all of its .ply files (or optionally pick a representative .ply per sample_id, depending on training strategy).

Model

• Backbone: PointNet++ (set abstraction + feature propagation)
• Head: Regression MLP to output a single scalar: predicted weight (grams or kilograms; keep consistent with labels).

Training (high level)

• Input: point cloud N x 3 (optionally N x 6 if adding normals)
• Preprocessing:
  • Sample a fixed number of points (e.g. 1024/2048)
  • Normalize/center (and optionally scale)
  • Augmentations (optional): jitter, small rotations, scaling
• Loss: L1 / SmoothL1 / MSE on weight
• Metrics: MAE (g), MAPE (%), correlation

Evaluation / comparison

The repo also supports exporting prediction/label pairs for comparison (e.g. JSON) so you can compute error statistics and analyze failure cases.

Known error sources & mitigations

1) Bad point clouds slipping through the “good/bad” classifier

Sometimes the point cloud quality classifier may classify a bad point cloud as good, causing unstable / biased weight predictions.

• Mitigation: increase the classifier confidence threshold so only higher-confidence “good” clouds are kept.
  • In the SVO2 pipeline this is controlled by --pointcloud-classifier-threshold (higher = stricter).
  • Trade-off: higher threshold usually reduces false-positives but may drop more clouds.

2) Segmentation mask includes fin/tail → point cloud becomes “fatter”

If the 2D mask is too large (e.g., fin or tail included), the depth->pointcloud will include non-body points and the fish appears larger/heavier than it actually is.

• Mitigation (recommended): label body-only masks and train a YOLOv8 segmentation model (YOLO-seg) so the mask excludes fins/tail.
  • See segmentation/README.md for the Labelme polygon → YOLOv8-seg dataset conversion and training scripts.

''' fish1/HD1080_SN43186771_11-21-55: avg=393.86 g (n=2) | actual=380.25 g | diff%=3.58% fish1/HD1080_SN43186771_11-30-54: avg=368.31 g (n=10) | actual=380.25 g | diff%=-3.14% fish1/HD1080_SN43186771_11-31-24: avg=399.35 g (n=53) | actual=380.25 g | diff%=5.02% fish1/HD1080_SN43186771_11-33-11: avg=426.33 g (n=2) | actual=380.25 g | diff%=12.12% fish10/HD1080_SN43186771_12-00-02: avg=403.65 g (n=116) | actual=336.00 g | diff%=20.13% fish11/HD1080_SN43186771_12-08-25: avg=434.81 g (n=16) | actual=339.50 g | diff%=28.07% fish11/HD1080_SN43186771_12-09-30: avg=380.54 g (n=26) | actual=339.50 g | diff%=12.09% fish12/HD1080_SN43186771_12-24-50: avg=523.37 g (n=148) | actual=574.50 g | diff%=-8.90% fish13/HD1080_SN43186771_12-35-18: avg=509.70 g (n=3) | actual=471.70 g | diff%=8.06% fish13/HD1080_SN43186771_12-40-42: avg=427.46 g (n=12) | actual=471.70 g | diff%=-9.38% fish13/HD1080_SN43186771_12-42-34: avg=418.53 g (n=13) | actual=471.70 g | diff%=-11.27% fish14/HD1080_SN43186771_12-48-46: avg=578.01 g (n=23) | actual=580.60 g | diff%=-0.45% fish14/HD1080_SN43186771_12-50-15: avg=511.41 g (n=54) | actual=580.60 g | diff%=-11.92% fish15/HD1080_SN43186771_12-58-43: avg=313.65 g (n=7) | actual=339.00 g | diff%=-7.48% fish15/HD1080_SN43186771_13-04-58: avg=316.70 g (n=5) | actual=339.00 g | diff%=-6.58% fish15/HD1080_SN43186771_13-06-21: avg=375.00 g (n=20) | actual=339.00 g | diff%=10.62% fish16/HD1080_SN43186771_13-11-14: avg=462.20 g (n=9) | actual=618.10 g | diff%=-25.22% fish16/HD1080_SN43186771_13-13-55: avg=505.61 g (n=95) | actual=618.10 g | diff%=-18.20% fish17/HD1080_SN43186771_13-25-55: avg=516.74 g (n=14) | actual=582.90 g | diff%=-11.35% fish18/HD1080_SN43186771_13-35-06: avg=357.36 g (n=9) | actual=414.00 g | diff%=-13.68% fish18/HD1080_SN43186771_13-35-34: avg=403.66 g (n=13) | actual=414.00 g | diff%=-2.50% fish18/HD1080_SN43186771_13-36-11: avg=432.68 g (n=2) | actual=414.00 g | diff%=4.51% fish19/HD1080_SN43186771_13-39-14: avg=355.39 g (n=34) | actual=333.50 g | diff%=6.56% fish2/HD1080_SN43186771_11-50-46: avg=451.50 g (n=6) | actual=487.45 g | diff%=-7.38% fish20/HD1080_SN43186771_13-47-05: avg=405.79 g (n=32) | actual=376.60 g | diff%=7.75% fish20/HD1080_SN43186771_13-48-39: avg=393.67 g (n=37) | actual=376.60 g | diff%=4.53% fish20/HD1080_SN43186771_13-51-13: avg=398.49 g (n=35) | actual=376.60 g | diff%=5.81% fish3/HD1080_SN43186771_12-11-17: avg=324.99 g (n=11) | actual=262.80 g | diff%=23.66% fish5/HD1080_SN43186771_10-20-27: avg=461.63 g (n=102) | actual=394.50 g | diff%=17.02% fish6/HD1080_SN43186771_10-27-17: avg=394.66 g (n=39) | actual=334.70 g | diff%=17.91% fish6/HD1080_SN43186771_10-32-11: avg=321.26 g (n=30) | actual=334.70 g | diff%=-4.02% fish7/HD1080_SN43186771_11-11-27: avg=392.47 g (n=101) | actual=333.90 g | diff%=17.54% fish8/HD1080_SN43186771_11-24-33: avg=498.19 g (n=59) | actual=616.80 g | diff%=-19.23% fish8/HD1080_SN43186771_11-33-46: avg=500.25 g (n=43) | actual=616.80 g | diff%=-18.90% fish8/HD1080_SN43186771_11-38-26: avg=584.64 g (n=18) | actual=616.80 g | diff%=-5.21% fish9/HD1080_SN43186771_11-45-30: avg=422.53 g (n=44) | actual=448.70 g | diff%=-5.83% Saved: /home/ubuntu/projects/FishMeasure/weight_estimator/fish_all20_predict.json '''

Error Analysis

1.case of fish 8 HD1080_SN43186771_11-33-26 the fish length is good, meaning the depth is good, the key observation is that the fish is closer and going downward.

HD1080_SN43186771_11-33-46 the fish length is not good, depth is not good, and the fish is on the top of the screen, that region has unstable depth 2.