From my experience estimates such as t-shirt sizes or story points should not directly be linked to concrete times because this mitigates their purpose. Typically their purpose is to focus on complexity of a task which can be estimates with relative to effort to provide a cost-efficient means to estimatation.
You could ask yourself what is the reason the team gives estimates in t-shirt sizes rather than man hours in your situation.
Would you gain the same benefits when directly giving an estimate as a range of hours? If this is the case, then just do this and do not bother to map these to t-shirt sizes. If this is not the case, you should just stick to t-shirt sizes and perform any additional considerations on top of that.
If the team gives estimates in t-shirt sizes, it is still possible to derive estimates on required effort or time to completion. However, this should be a process separate from the estimation of complexity, as this burden cannot be put upon the team without mitigating the advantages of the estimation process.
It should rather be seen as a forecasting process that takes into account certain empirical values such as the average time tasks estimated with certain t-shirt sizes took to complete in the past and provides a forecast based on this data.
The main point is to provide an environment in which the team gives estimates based solely on their true understanding of relative sizes rather than being influenced by project-management considerations.
If separated in this way, neither the estimates nor the forecast can be really wrong. Of course the team might be wrong on their estimate of relative sizes but this is to be expected due to the limited knowledge at the time of estimation which lies in the very nature of all estimation activities. The actually required effort or time to complete a task might also differ from the forecast, but, again, this lies in the nature of forecasting ans its probabilistic nature. The best you can aim for is reach a high likelihood rather than a low one.
What you can tell the customer is that based on your past experience with tasks whose complexity has been estimated similarly, that it will take a certain amount of effort, or more precisely you should provide a likelihood that it will fall into a certain range of effort. Here, you can use a wide range of estimation techniques and statistical methods.
From my experience, it can be worthwile to go through the difficulty of trying to clearly communicate the nature of estimation of complexity and forecasting effort to set up realistic expectations. The message should be that you have a common interest in forecasting requires effort so that the customer can take informed decisions. A situation in which effort estimates are negotiated between to opposing parties should be avoided.