Non-price attributes such as prior relationship, product quality, and reliability can be more important than bidding prices for the buyers when selecting the winner of a reverse auction. In this regard, in open-ended reverse auctions (also known as buyer-determined auctions), buyers usually evaluate and assign 'quality score' (QS) to each supplier. Although QS enables the buyer to incorporate the above-mentioned non-price attributes in its sourcing decisions, it may have repercussions on the outcome of a reverse auction. On one hand, the QS influences the relative positions of the suppliers against each other in terms of the buyer's sourcing preference. On the other hand, this very characteristics of QS may invalidate its role among the suppliers as the buyer may become tempted to abuse it in order to regulate the competition towards its own benefits. In order to analyze the tug of war between the operational and strategic benefits of QS in a reverse auction setting, in this paper, we develop a bi-level supply chain between a buyer and two competing suppliers. One of the two suppliers is an incumbent with known track record, whereas the other one is untested. Ceteris paribus, the buyer calculates the (relative) QS of untested supplier with respect to the known one, and decides whether or not (and if yes, how) to credibly share this with the two suppliers possibly via advance minimum revenue guarantees. Analyzing pooling, separating, and semi-separating equilibria of resulting signalling game between the buyer and the two suppliers, we develop insights on the impact of QS on the buyer-determined reverse (procurement) auctions. Our results suggest that such advance guarantees should be offered only to the incumbent supplier when the difference between suppliers' quality scores is relatively low. In addition, the degree of price competition among suppliers increases when the degree of information asymmetry between the upstream and downstream levels of supply chain regarding the quality scores is sufficiently low. Moreover, when the number of qualified entrants in the auction increases, suppliers' price competition and buyer's signaling cost increase, which suggests the buyers not to share the quality scores information. We have provided the managerial implications of our findings.