Additionally, is it feasible to evaluate our submission only on the metrics that a company reports? This approach would enable us to better assess the accuracy of our models without the boost by the current 0 values.

This would change my current approach (and I assume others) quite significantly. At the moment, I have designed my RAG system such that it should return 'null' (which is mapped to 0) if the metric isn't found in the pdf, and returns the value of the metric if it is.

Are others just using their model to retrieve the metrics we know each company reports (by looking at which metrics the company reported in previous years), and leaving the rest as 0? And is this allowed? Obviously, you'll get a higher score by not using your model to try and retrieve metrics that aren't reported (as sometimes the model will incorrectly return a value instead of 'null'/0 in this case). But it makes the model practically less useful, as in practise you won't know in advance whether a company reports a specific metric or not, so the model should be able to handle this case correctly (return 'null' if not found).