To me, this may be the most important point of them all, and I really appreciate you spelling it out here.
While for example ‘number of users’ may be an appropriate metric for comparing otherwise similar analytics services, all of which target highly skilled investors, it could be a very bad idea to use that same metric indiscriminately for general educational or news services - as that might mean rewarding the most attention grabbing or addictive approaches over those that create high quality content for specific target groups.
If we want Ether’s Phoenix to be in the service of humanity, we really need to think hard about the relationship between quantitative and qualitative metrics.
A good rule of thumb might be that the more we care about certain qualitative metrics (in the context of certain project categories), the more important it is to make sure that they are incorporated into the impact evaluation.
And the more a project category targets / affects ‘the whole human’, the more we should probably require subjective human judgement to be a crucial evaluation criteria.
To the extend that human experience is the goal, human experience should also be the measuring stick.