Start with roles, not volume
Choose the references that explain your subject, environment and motion. Give each reference a clear name. In your prompt, explicitly say which reference controls each creative decision. More references do not automatically mean a more consistent result.
Save the decision with the result
Keep the prompt version, ordered reference list, model and parameters with each job. When a shot works, you can identify what produced it. When it fails, change one variable and submit a new job rather than overwriting your history.