Thursday, May 9, 2019

assembly - What does `rep ret` mean?



I was testing some code on Visual Studio 2008 and noticed security_cookie. I can understand the point of it, but I don't understand what the purpose of this instruction is.



    rep ret /* REP to avoid AMD branch prediction penalty */


Of course I can understand the comment :) but what is this prefix exaclty doing in context with the ret and what happens if ecx is != 0? Apparently the loop count from ecx is ignored when I debug it, which is to be expected.




The code where I found this was here (injected by the compiler for security):



void __declspec(naked) __fastcall __security_check_cookie(UINT_PTR cookie)
{
/* x86 version written in asm to preserve all regs */
__asm {
cmp ecx, __security_cookie
jne failure
rep ret /* REP to avoid AMD branch prediction penalty */
failure:

jmp __report_gsfailure
}
}

Answer



There's a whole blog named after this instruction. And the first post describes the reason behind it: http://repzret.org/p/repzret/



Basically, there was an issue in the AMD's branch predictor when a single-byte ret immediately followed a conditional jump as in the code you quoted (and a few other situations), and the workaround was to add the rep prefix, which is ignored by CPU but fixes the predictor penalty.


No comments:

Post a Comment

plot explanation - Why did Peaches' mom hang on the tree? - Movies & TV

In the middle of the movie Ice Age: Continental Drift Peaches' mom asked Peaches to go to sleep. Then, she hung on the tree. This parti...