# Fish Weight Prediction – Ideas

## Multi-modal weight prediction (image + point cloud)

**Idea:** Instead of using only PointNet++ to predict weight from the 3D point cloud, use two separate models and combine their outputs:

1. **Image-based weight predictor** – A model that predicts fish weight from 2D images (e.g., RGB frames from the video).
2. **Point cloud–based weight predictor** – The existing PointNet++ model that predicts weight from 3D point clouds.

Then **combine the two feature representations** (or predictions) to produce a final weight estimate. This could be done by:

- Concatenating features from both encoders and feeding them into a small fusion head.
- Averaging or otherwise combining the two predictions (e.g., weighted average).
- Using a learned fusion module that decides how much to trust each modality.

**Rationale:** 2D images and 3D point clouds provide complementary information. Images capture texture, color, and fine visual details; point clouds capture geometry and scale. Combining both may improve robustness when one modality is noisy or incomplete (e.g., poor depth, occlusion, or low-quality segmentation).