2. Analog vs Digital

The real world is mostly analog: sound, light, and temperature change continuously. Computers work digitally by representing everything as numbers. Before an NLP model can understand Arabic text, every character must become numerical data.

Arabic characters become numbers

text = "سلام"

for c in text:
    print(c, ord(c))

Output

س 1587
ل 1604
ا 1575
م 1605

Numbers become binary

letter = "س"

print(ord(letter))
print(bin(ord(letter)))

Output

1587
0b11000110011

UTF-8 bytes

text = "سلام"

print(text.encode("utf-8"))

Output

b'\xd8\xb3\xd9\x84\xd8\xa7\xd9\x85'