RSIM uses limited buffers in order to accurately simulate contention for resources at the various modules, raising the possibility of deadlock in the system. To avoid deadlock, requests and replies are kept physically separate. Conceptually, requests are messages that allocate resources (such as MSHRs, pending coherence bits, or directory buffer entries). Replies conceptually release resources, and must be guaranteed to be accepted in a finite amount of time, even if they lead to new messages. This is achieved by allocating a resource for the reply at the time the request is sent out. Thus, the reply is guaranteed to have an associated resource waiting for it when it arrives. The incoming reply can be accepted by the module immediately and held in that resource until the completion of any additional processing it may require.
In our system, for deadlock avoidance purposes, requests include REQUEST and COHE messages, while replies include REPLY and COHE_REPLY (including write-back and replacement) messages. Although write-backs and replacement messages are unsolicited and do not already have resources reserved for them in the directory and memory module, they can be considered replies because they do not require additional resources or send out additional messages. (In our system, write-backs do not receive acknowledgments from the memory controller. If such acknowledgments are required, write-backs should be sent as requests with a resource held at the cache until the acknowledgment arrives.)
Each subsystem in the RSIM memory system simulator takes specific precautions to prevent the creation of new deadlocks. These steps are described separately with each subsystem.