View Full Version : AceComm & AceSerializer -> Eluna

11-29-2014, 02:02 PM
AceComm and AceSerializer work in combination perfectly to send large objects (over 255 bytes) across the WoW addon channel.
AceComm handles splitting and joining of messages, AceSerializer handles converting objects (all except functions) <-> strings (which can then be sent using AceComm).

Here are the code files:

AceComm = {}
-- Lua APIs
local type, next, pairs, tostring = type, next, pairs, tostring
local strsub, strfind = string.sub, string.find
local match = string.match
local tinsert, tconcat = table.insert, table.concat
local error, assert = error, assert

-- Global vars/functions that we don't upvalue since they might get hooked, or upgraded
-- List them here for Mikk's FindGlobals script
-- GLOBALS: LibStub, DEFAULT_CHAT_FRAME, geterrorhandler, RegisterAddonMessagePrefix

AceComm.embeds = AceComm.embeds or {}

-- for my sanity and yours, let's give the message type bytes some names
local MSG_MULTI_FIRST = "\001"
local MSG_MULTI_NEXT = "\002"
local MSG_MULTI_LAST = "\003"
local MSG_ESCAPE = "\004"

-- remove old structures (pre WoW 4.0)
AceComm.multipart_origprefixes = nil
AceComm.multipart_reassemblers = nil

-- the multipart message spool: indexed by a combination of sender+distribution+
AceComm.multipart_spool = AceComm.multipart_spool or {}

--- Register for Addon Traffic on a specified prefix
-- @param prefix A printable character (\032-\255) classification of the message (typically AddonName or AddonNameEvent), max 16 characters
-- @param method Callback to call on message reception: Function reference, or method name (string) to call on self. Defaults to "OnCommReceived"
function AceComm:RegisterComm(prefix, method)
if method == nil then
method = "OnCommReceived"

if #prefix > 16 then -- TODO: 15?
error("AceComm:RegisterComm(prefix,method): prefix length is limited to 16 characters")
local callbacks = AceComm.callbacks[prefix] or {}
tinsert(callbacks, method);
AceComm.callbacks[prefix] = callbacks;

local warnedPrefix=false

--- Send a message over the Addon Channel
-- @param prefix A printable character (\032-\255) classification of the message (typically AddonName or AddonNameEvent)
-- @param text Data to send, nils (\000) not allowed. Any length.
-- @param distribution Addon channel number reference.
-- @param target Destination for some distributions; see SendAddonMessage API
function AceComm:SendCommMessage(prefix, text, distribution, target)
local player = Global:GetPlayerByName(target);
if player == nil then

local textlen = #text
local maxtextlen = 255 -- Yes, the max is 255 even if the dev post said 256. I tested. Char 256+ get silently truncated. /Mikk, 20110327
local queueName = prefix..distribution..(target or "")

local forceMultipart
if match(text, "^[\001-\009]") then -- 4.1+: see if the first character is a control character
-- we need to escape the first character with a \004
if textlen+1 > maxtextlen then -- would we go over the size limit?
forceMultipart = true -- just make it multipart, no escape problems then
text = "\004" .. text

if not forceMultipart and textlen <= maxtextlen then
-- fits all in one message
player:SendAddonMessage(prefix, text, distribution, player)
maxtextlen = maxtextlen - 1 -- 1 extra byte for part indicator in prefix(4.0)/start of message(4.1)

-- first part
local chunk = strsub(text, 1, maxtextlen)
player:SendAddonMessage(prefix, MSG_MULTI_FIRST..chunk, distribution, player)
-- continuation
local pos = 1+maxtextlen

while pos+maxtextlen <= textlen do
chunk = strsub(text, pos, pos+maxtextlen-1)
player:SendAddonMessage(prefix, MSG_MULTI_NEXT..chunk, distribution, player)
pos = pos + maxtextlen

-- final part
chunk = strsub(text, pos)
player:SendAddonMessage(prefix, MSG_MULTI_LAST..chunk, distribution, player)

-- Message receiving

local compost = setmetatable({}, {__mode = "k"})
local function new()
local t = next(compost)
if t then
for i=#t,3,-1 do -- faster than pairs loop. don't even nil out 1/2 since they'll be overwritten
return t

return {}

local function lostdatawarning(prefix,sender,where)
DEFAULT_CHAT_FRAME:AddMessage(MAJOR..": Warning: lost network data regarding '"..tostring(prefix).."' from '"..tostring(sender).."' (in "..where..")")

function AceComm:OnReceiveMultipartFirst(prefix, message, distribution, sender)
local key = prefix.."\t"..distribution.."\t"..sender -- a unique stream is defined by the prefix + distribution + sender
local spool = AceComm.multipart_spool

if spool[key] then
-- continue and overwrite

spool[key] = message -- plain string for now

function AceComm:OnReceiveMultipartNext(prefix, message, distribution, sender)
local key = prefix.."\t"..distribution.."\t"..sender -- a unique stream is defined by the prefix + distribution + sender
local spool = AceComm.multipart_spool
local olddata = spool[key]

if not olddata then

if type(olddata)~="table" then
-- ... but what we have is not a table. So make it one. (Pull a composted one if available)
local t = new()
t[1] = olddata -- add old data as first string
t[2] = message -- and new message as second string
spool[key] = t -- and put the table in the spool instead of the old string
tinsert(olddata, message)

function AceComm:OnReceiveMultipartLast(prefix, message, distribution, sender, player)
local key = prefix.."\t"..distribution.."\t"..sender -- a unique stream is defined by the prefix + distribution + sender
local spool = AceComm.multipart_spool
local olddata = spool[key]

if not olddata then

spool[key] = nil

if type(olddata) == "table" then
-- if we've received a "next", the spooled data will be a table for rapid & garbage-free tconcat
tinsert(olddata, message)
for k,v in pairs(AceComm.callbacks[prefix]) do
v(prefix, tconcat(olddata, ""), distribution, sender, player)
compost[olddata] = true
-- if we've only received a "first", the spooled data will still only be a string
for k,v in pairs(AceComm.callbacks[prefix]) do
v(prefix, olddata..message, distribution, sender, player)

-- Embed CallbackHandler
AceComm.callbacks = {}

local function OnEvent(event, player, distribution, prefix, message, target)
--if target == "SERVER" then
local sender = player:GetName()
sender = sender or "none"
local control, rest = match(message, "^([\001-\009])(.*)")
if control then
if control==MSG_MULTI_FIRST then
AceComm:OnReceiveMultipartFirst(prefix, rest, distribution, sender)
elseif control==MSG_MULTI_NEXT then
AceComm:OnReceiveMultipartNext(prefix, rest, distribution, sender)
elseif control==MSG_MULTI_LAST then
AceComm:OnReceiveMultipartLast(prefix, rest, distribution, sender, player)
elseif control==MSG_ESCAPE then
for k,v in pairs(AceComm.callbacks[prefix]) do
v(prefix, rest, distribution, sender, player)
-- unknown control character, ignore SILENTLY (dont warn unnecessarily about future extensions!)
for k,v in pairs(AceComm.callbacks[prefix]) do
v(prefix, message, distribution, sender, player)

RegisterServerEvent(30, OnEvent)


AceSerializer = {}

-- Lua APIs
local strbyte, strchar, gsub, gmatch, format = string.byte, string.char, string.gsub, string.gmatch, string.format
local assert, error, pcall = assert, error, pcall
local type, tostring, tonumber = type, tostring, tonumber
local pairs, select, frexp = pairs, select, math.frexp
local tconcat = table.concat

-- quick copies of string representations of wonky numbers
local inf = math.huge

local serNaN -- can't do this in 4.3, see ace3 ticket 268
local serInf = tostring(inf)
local serNegInf = tostring(-inf)

-- Serialization functions

local function SerializeStringHelper(ch) -- Used by SerializeValue for strings
-- We use \126 ("~") as an escape character for all nonprints plus a few more
local n = strbyte(ch)
if n==30 then -- v3 / ticket 115: catch a nonprint that ends up being "~^" when encoded... DOH
return "\126\122"
elseif n<=32 then -- nonprint + space
return "\126"..strchar(n+64)
elseif n==94 then -- value separator
return "\126\125"
elseif n==126 then -- our own escape character
return "\126\124"
elseif n==127 then -- nonprint (DEL)
return "\126\123"
assert(false) -- can't be reached if caller uses a sane regex

local function SerializeValue(v, res, nres)
-- We use "^" as a value separator, followed by one byte for type indicator
local t=type(v)

if t=="string" then -- ^S = string (escaped to remove nonprints, "^"s, etc)
res[nres+1] = "^S"
res[nres+2] = gsub(v,"[%c \94\126\127]", SerializeStringHelper)

elseif t=="number" then -- ^N = number (just tostring()ed) or ^F (float components)
local str = tostring(v)
if tonumber(str)==v --[[not in 4.3 or str==serNaN]] or str==serInf or str==serNegInf then
-- translates just fine, transmit as-is
res[nres+1] = "^N"
res[nres+2] = str
local m,e = frexp(v)
res[nres+1] = "^F"
res[nres+2] = format("%.0f",m*2^53) -- force mantissa to become integer (it's originally 0.5--0.9999)
res[nres+3] = "^f"
res[nres+4] = tostring(e-53) -- adjust exponent to counteract mantissa manipulation

elseif t=="table" then -- ^T...^t = table (list of key,value pairs)
res[nres] = "^T"
for k,v in pairs(v) do
nres = SerializeValue(k, res, nres)
nres = SerializeValue(v, res, nres)
res[nres] = "^t"

elseif t=="boolean" then -- ^B = true, ^b = false
if v then
res[nres] = "^B" -- true
res[nres] = "^b" -- false

elseif t=="nil" then -- ^Z = nil (zero, "N" was taken :P)
res[nres] = "^Z"

error(MAJOR..": Cannot serialize a value of type '"..t.."'") -- can't produce error on right level, this is wildly recursive

return nres

local serializeTbl = { "^1" } -- "^1" = Hi, I'm data serialized by AceSerializer protocol rev 1

--- Serialize the data passed into the function.
-- Takes a list of values (strings, numbers, booleans, nils, tables)
-- and returns it in serialized form (a string).\\
-- May throw errors on invalid data types.
-- @param ... List of values to serialize
-- @return The data in its serialized form (string)
function AceSerializer:Serialize(...)
local nres = 1

for i=1,select("#", ...) do
local v = select(i, ...)
nres = SerializeValue(v, serializeTbl, nres)

serializeTbl[nres+1] = "^^" -- "^^" = End of serialized data

return tconcat(serializeTbl, "", 1, nres+1)

-- Deserialization functions
local function DeserializeStringHelper(escape)
if escape<"~\122" then
return strchar(strbyte(escape,2,2)-64)
elseif escape=="~\122" then -- v3 / ticket 115: special case encode since 30+64=94 ("^") - OOPS.
return "\030"
elseif escape=="~\123" then
return "\127"
elseif escape=="~\124" then
return "\126"
elseif escape=="~\125" then
return "\94"
error("DeserializeStringHelper got called for '"..escape.."'?!?") -- can't be reached unless regex is screwed up

local function DeserializeNumberHelper(number)
--[[ not in 4.3 if number == serNaN then
return 0/0
else]]if number == serNegInf then
return -inf
elseif number == serInf then
return inf
return tonumber(number)

-- DeserializeValue: worker function for :Deserialize()
-- It works in two modes:
-- Main (top-level) mode: Deserialize a list of values and return them all
-- Recursive (table) mode: Deserialize only a single value (_may_ of course be another table with lots of subvalues in it)
-- The function _always_ works recursively due to having to build a list of values to return
-- Callers are expected to pcall(DeserializeValue) to trap errors

local function DeserializeValue(iter,single,ctl,data)

if not single then
ctl,data = iter()

if not ctl then
error("Supplied data misses AceSerializer terminator ('^^')")

if ctl=="^^" then
-- ignore extraneous data

local res

if ctl=="^S" then
res = gsub(data, "~.", DeserializeStringHelper)
elseif ctl=="^N" then
res = DeserializeNumberHelper(data)
if not res then
error("Invalid serialized number: '"..tostring(data).."'")
elseif ctl=="^F" then -- ^F<mantissa>^f<exponent>
local ctl2,e = iter()
if ctl2~="^f" then
error("Invalid serialized floating-point number, expected '^f', not '"..tostring(ctl2).."'")
local m=tonumber(data)
if not (m and e) then
error("Invalid serialized floating-point number, expected mantissa and exponent, got '"..tostring(m).."' and '"..tostring(e).."'")
res = m*(2^e)
elseif ctl=="^B" then -- yeah yeah ignore data portion
res = true
elseif ctl=="^b" then -- yeah yeah ignore data portion
res = false
elseif ctl=="^Z" then -- yeah yeah ignore data portion
res = nil
elseif ctl=="^T" then
-- ignore ^T's data, future extensibility?
res = {}
local k,v
while true do
ctl,data = iter()
if ctl=="^t" then break end -- ignore ^t's data
k = DeserializeValue(iter,true,ctl,data)
if k==nil then
error("Invalid AceSerializer table format (no table end marker)")
ctl,data = iter()
v = DeserializeValue(iter,true,ctl,data)
if v==nil then
error("Invalid AceSerializer table format (no table end marker)")
error("Invalid AceSerializer control code '"..ctl.."'")

if not single then
return res,DeserializeValue(iter)
return res

--- Deserializes the data into its original values.
-- Accepts serialized data, ignoring all control characters and whitespace.
-- @param str The serialized data (from :Serialize)
-- @return true followed by a list of values, OR false followed by an error message
function AceSerializer:Deserialize(str)
str = gsub(str, "[%c ]", "") -- ignore all control characters; nice for embedding in email and stuff

local iter = gmatch(str, "(^.)([^^]*)") -- Any ^x followed by string of non-^
local ctl,data = iter()
if not ctl or ctl~="^1" then
-- we purposefully ignore the data portion of the start code, it can be used as an extension mechanism
return false, "Supplied data is not AceSerializer data (rev 1)"

return pcall(DeserializeValue, iter)

I've modified the usage (and a bit of the inner workings) of the comm addon to make it more suited to Eluna and a server environment.

Here's an example usage of communication(ported directly from the addon communication thread):

local MName = "[ServerAddonHandler]"
print(MName.." Loaded")
local pfx = "Eluna"
function OnReceiveAddonMsg(prefix, message, dist, sender, pSender)
if (message == "Ping") then
print(MName..": Ping received by "..sender.."! Sending Pong.")
AceComm:SendCommMessage(prefix, "Pong", dist, sender)
elseif(message == "GoldRequest") then
print(MName..": Gold request received from Client.")
print(MName..": Received unknown msg: "..message)
AceComm:RegisterComm(pfx, OnReceiveAddonMsg)

This does the same thing as the original code. There's no advantage beyond not having to check the prefix (as this handler only handles 1 prefix). However, if the words "Ping" or "Pong" (the message payload) were to exceed 255 characters, then the entire message would be truncated to 255 characters.
If the addon uses the AceComm library to send the message, however: Removed
It will be split up into several 255 byte messages and reassembled at the server end.

Here's an example usage of AceSerializer:

local serialized = AceSerializer:Serialize("Ohai", "Der", "Eluna");
local success, a, b, c = AceSerializer:Deserialize(serialized);
print(a.." "..b.." "..c);

11-29-2014, 03:24 PM
This is actually 3rd system like this I have seen so far for Eluna. :3
In case you are interested in the others, they are GATE (https://github.com/ElunaLuaEngine/ElunaGate) and AIO (https://github.com/Rochet2/AIO)

The main differences would be that they have ready coded functionality to send longer messages than 255 characters ~Though after reading it through it seems this is implemented?
and the communication system is used to achieve creating addons form server side, so the messaging system is a side effect rather than the actual mission of the code.

I made a quick read through the code since I was interested in how the inner workings are implemented.

I suggest that this isnt the default way to access the target: local player = Global:GetPlayerByName(target)
Rather pass the player object as target. The packet sender can then decide whether to get the player by name or other.
Fetching players globally by name is the slowest method. Also that is not necessary if the player object was available in any case already.
Elsehwere the sender name is also only passed when the sender player could be passed.
Also do note that it is actually GetPlayerByName(target), without the Global.

The code doesnt seem to limit the amount of messages from client side. This can possibly lead to someone sending an endless message to server hogging memory. Maybe.
However for security reasons, at least on AIO, this is limited to a number of messages (changeable) so an user cant hang the server.
Also a bit vary of executing pcall on serverside on client's data.

The API info in wowwiki (http://www.wowwiki.com/API_SendAddonMessage#toc)said that the prefix and \t were included in the msg length.
However interestingly this implementation ignores the prefix length and uses 255 as max length for the total message. I assume this was Ace's code, so makes me wonder a bit considering it should have plenty users ~ so it should be ok..

12-02-2014, 09:54 AM
It seems you're right! This was a port of the WoD Ace library. 3.3.5a includes the prefix in the message length. If only there were some way to send an unlimited amount of data to the server from the UI.... :L