The absolute most important metric would be the number of database queries executed. If the ORM's translation of the logical code result in a different number of queries, this will have an effect that will dwarf any other difference. You should be especially on the lookout for "n+1"-pattern where the ORM will perform an individual query for each item in a collection when you traverse sub-objects. At least in EF you often have the choice between eager and lazy (n+1) loading of sub-objects, so be sure you know how to use the ORM optimally before dismissing one or the other.
Second, you should look at the amount of data returned from the database. There could be a difference if one ORM performs some filtering on the client side and another does in the sql. Some ORMS are smarter than others in how they translate filters to sql. Obviously it is preferable to filter as much as possible on the database side.
If the ORM's on the other hand generates equivalent sql (which is not unlikely), the performance difference will likely be small, maybe even negligible, and you should base your choice on other factors like maintainability and developer familiarity.