Erlang通用二进制格式

UBF是一个让Erlang和外部世界交互的一个框架. 文档和相应的开源库基于Joe Armstrong最初的UBF站点和代码,增加了MIT许可文件,增加了大量的增强和改进.

UBF是一个跨网络转换和描述复杂数据结构的语言. 它包括三个部分:

  • UBF(a) 是一个语言中性的数据描述格式, 粗略地等同于具有良好格式的XML.
  • UBF(b) 是一个描述性的编程语言, 它用于描述UBF(a)中的类型, 以及客户端和服务器之间的协议.
  • UBF(c) 是一个在UBF客户端和UBF服务器之间使用的低级协议(meta-level).

UBF设计用于生产级别的部署和要求24x7x365可靠性的电信级系统.

什么是语言中性?

这是相对于语言独立的二进制格式而言, 它的实现是与特定的语言相关的, 支持的格式定义语言如下:

  • thrift
  • redis
  • jsonrpc
  • eep8
  • bertrpc
  • abnf

可以在项目首页看到多种格式的实现.

参考资料

  1. Erlang Universal Binary Format? Anyone using it?
    http://stackoverflow.com/questions/4731449/erlang-universal-binary-format-anyone-using-it
  2. 项目地址
    https://github.com/ubf/ubf
  3. UBF用户指南
    http://ubf.github.io/ubf/ubf-user-guide.en.html

XMPP Strophe.js插件 - 带内注册

客户端服务器交互

下图,是注册过程客户端和服务器的交互过程, 绿色的输出是客户端请求的XML节, 蓝色的输出是服务器应答的XML节.

XMPP Stanzas

下载插件

https://github.com/strophe/strophejs-plugins/tree/master/register

HTML页面代码

创建两个表单元素username和password用于获取用户名和密码, 一个Register按钮触发注册操作.
本文所演示示例的完整代码在此: https://gist.github.com/developerworks/317ccf6eb2d3060610f8

1
<!doctype html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Register a user</title>
  <!--// JQuery库-->
  <script type="text/javascript" src="jquery.min.js"></script>
  <!--// Strophe.js库-->
  <script type="text/javascript" src="strophe.js"></script>
  <!--// Strophe.js 注册插件-->
  <script type="text/javascript" src="strophe.register.js"></script>
  <!--// 注册业务逻辑-->
  <script type="text/javascript" src="register.js"></script>
</head>
<body>
  Username:<input type="text" id="username" placeholder="Please input username"/>
  Password<input type="text" id="password" placeholder="Please input your password"/>
  <br/>
  <button id="register">Register</button>
</form>
</body>
</html>

回调函数

register.js

1
// 调试用,XML格式化函数
function formatXml(xml) {
  var formatted = '';
  var reg = /(>)(<)(\/*)/g;
  xml = xml.replace(reg, '$1\r\n$2$3');
  var pad = 0;
  jQuery.each(xml.split('\r\n'), function (index, node) {
    var indent = 0;
    if (node.match(/.+<\/\w[^>]*>$/)) {
      indent = 0;
    } else if (node.match(/^<\/\w/)) {
      if (pad != 0) {
        pad -= 1;
      }
    } else if (node.match(/^<\w[^>]*[^\/]>.*$/)) {
      indent = 1;
    } else {
      indent = 0;
    }
    var padding = '';
    for (var i = 0; i < pad; i++) {
      padding += '  ';
    }
    formatted += padding + node + '\r\n';
    pad += indent;
  });
  return formatted;
}
// Websocket端点
var BOSH_SERVICE = 'ws://xmpp.myserver.info:5288';
// 域名
var DOMAIN_NAME = 'xmpp.myserver.info';
// XMPP连接对象
var connection = null;
// 浏览器控制台日志
function log(msg, sent) {
  if (sent) {
    console.log("%c\n" + msg, "color:green;");
  } else {
    console.log("%c\n" + msg, "color:blue;");
  }
}
function rawInput(data) {
  log(formatXml(data), false);
}
function rawOutput(data) {
  log(formatXml(data), true);
}
$(document).ready(function () {
  $('#register').bind('click', function () {
    // 创建连接
    connection = new Strophe.Connection(BOSH_SERVICE);
    // 注册事件处理器
    connection.rawInput = rawInput;
    connection.rawOutput = rawOutput;
    // 用户注册
    connection.register.connect(DOMAIN_NAME, function (status) {
      if (status === Strophe.Status.REGISTER) {
        connection.register.fields.username = $("#username").val();
        connection.register.fields.password = $("#password").val();
        console.info("registering...");
        connection.register.submit();
      }
      else if (status === Strophe.Status.REGISTERED) {
        console.info("Registerd!");
        connection.disconnect();
      }
      else if (status === Strophe.Status.CONFLICT) {
        console.error("Username already exists.")
      }
      else if (status === Strophe.Status.NOTACCEPTABLE) {
        console.error("Registration form not properly filled out.")
      }
      else if (status === Strophe.Status.REGIFAIL) {
        console.log("The Server does not support In-Band Registration")
      }
      else if (status === Strophe.Status.CONNECTED) {
        console.info("Connected.")
      }
      else if (status === Strophe.Status.DISCONNECTING) {
        console.log("Disconneting...");
      }
      else if (status === Strophe.Status.DISCONNECTED) {
        console.log("Disconneted.")
      }
      else {
      }
    }, 60, 1);
  });
});

XMPP XEP-0198流管理 - 协议

本文包括两个部分

介绍

XMPP Core 用XMPP定义了流的XML技术(也就是流的建立和终止,包括认证和加密).但是核心的XMPP协议并没有为管理一个灵活的XML流提供工具.

流管理背后的基本概念是,初始化的实体(一个服务端或者客户端)和接收的实体(一个服务端)可以为更灵活的管理stream交换命令.下面两条流管理的特性被广泛的关注,因为它们可以提高网络的可靠性和终端用户的体验:

  • Stanza确认(Stanza Acknowledgements) – 能够确认一段或者一系列Stanza是否已被某一方接收.
  • 流恢复(Stream Resumption) – 能够迅速的恢复(resume)一个已经被终止的流.

流管理用较的短XML元素实现了这些特性,这些XML元素是在流的标准上的.这些元素并不是XMPP意义上的stanzas(也就是说,不是<iq/>,<message/>,或<presence/>这样的stanzas,stanzas在RFC 6120中有定义),它们不会在流管理中被counted或者被acked,因为它们是为管理stanzas本身而存在的.

流管理是在XML流的标准上使用的.检查一个给定的流TCP的连通性的时候,特别推荐使用whitespace keepalives(见RFC 6120),XMPP Ping (XEP-0199)或者TCP keepalives.对比流管理,高级消息处理Advanced Message Processing (XEP-0079)和消息回执Message Delivery Receipts (XEP-0184),定义了Ack,它可以通过多个流实现端对端的传输;这些特性在一些特殊情况中是有用的,但是没必要去检查在两个xmpp实体之间直接传递的流.

注:流管理可以用于服务端到服务端,客户端到服务端的流.但是,为了方便,本规范只讨论客户端到服务端的流.同样的原则也适用于服务器到服务器的流.(在本文档中,以C:开头的都是由客户端发送的,由S:开头的都是由服务器发送的).

XMPP XEP-0198流管理 - 实例分析

客户端启用流管理

1
<enable xmlns='urn:xmpp:sm:3' resume='true'/>

服务端启用流管理

1
<enabled xmlns="urn:xmpp:sm:3"
    id="g2gCbQAAABkyNTY2MjI4NzA1MTQxMjIwMTU0MjEwMjkzaANiAAAFhGIAAE65YgAMkl8="
    resume="true" max="300"
    xmlns:stream="http://etherx.jabber.org/streams" version="1.0"/>

Strophe.js代码

1
var enable = function () {
  var stanza = $build('enable', {xmlns: 'urn:xmpp:sm:3', resume: true});
  connection.send(stanza.tree());
};

服务器日志

1
2014-09-29 22:12:39.416 [info] <0.14329.0>@ejabberd_c2s:handle_enable:2676
    Stream management with resumption enabled for root@xmpp.myserver.info/2619252428141228749171506

当启用流管理功能后,客户端在发送和接受每一个XML节的时候,会附带收到<r/>请求

1
<r xmlns="urn:xmpp:sm:3" xmlns:stream="http://etherx.jabber.org/streams" version="1.0"/>

Erlang MySQL驱动

编译

1
git clone git://github.com/dizzyd/erlang-mysql-driver.git erlang-mysql-driver.git
cd erlang-mysql-driver
wget -O ./build.sh http://svn.process-one.net/ejabberd-modules/mysql/trunk/build.sh
chmod +x ./build.sh
./build.sh

编译erlang-mysql-driver

Examples

1
%% Author: Administrator
%% Created: 2011-5-15
%% Description: TODO: Add description to test
-module(test).
%%
%% Include files
%%
%%
%% Exported Functions
%%
-export([start/0]).
%%
%% API Functions
%%
%%
%% Local Functions
%%
start() ->
%% Build connection
%% 注意最后一个参数,utf8,支持中文必杀技.
    mysql:start_link(connection,"localhost", 3306,"root", "root", "erlang", undefined, utf8),
%% INSERT
    _result = mysql:fetch(connection, [<<"INSERT INTO test_table (id, first_name, last_name) VALUES(default, '你好', '世界')">>]),
    io:format("Result1: ~p~n", [_result]),
%% SELECT
    _result2 = mysql:fetch(connection, [<<"SELECT * FROM test_table;">>]),
    io:format("Result2: ~p~n", [_result2]),
%% UPDATE
    _result3 = mysql:fetch(connection, [<<"UPDATE test_table SET first_name = 'River' WHERE first_name = 'ZHIQIANG' AND last_name = 'HE';">>]),
    io:format("Result3: ~p~n", [_result3]),
%% SELECT
    _result4 = mysql:fetch(connection, [<<"SELECT * FROM test_table;">>]),
    io:format("Result4: ~p~n", [_result4]),
%% DELETE
    _result5 = mysql:fetch(connection, [<<"DELETE FROM test_table WHERE last_name = 'HE' AND first_name = 'River';">>]),
    io:format("Result5: ~p~n", [_result5]),
%% SELECT
    _result6 = mysql:fetch(connection, [<<"SELECT * FROM test_table;">>]),
    io:format("Result6: ~p~n", [_result6]),
    ok.

Ejabberd 发布订阅实验

环境

Software Version
Ejabberd 14.07
Strophe.js 1.1.3

创建节点

Strophe.js代码

1
var createnode = function (node) {
  var iq = $iq({
    type: 'set',
    from: jid,
    to: session.pubsub,
    id: getId()
  }).c('pubsub', {xmlns: 'http://jabber.org/protocol/pubsub'})
    .c('create', {node: node});
  connection.send(iq.tree());
};

XML节

1
<!--客户端IQ-set请求-->
<iq type='set'
    from='root@xmpp.myserver.info'
    to='pubsub.xmpp.myserver.info'
    id='id-1412188942892'
    xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <create node='/home/test2'/>
  </pubsub>
</iq>
<!--服务端应答IQ-result-->
<iq from="pubsub.xmpp.myserver.info"
    to="root@xmpp.myserver.info/17735350171411959891145164"
    id="id-1412188942892"
    type="result"
    xmlns="jabber:client"
    xmlns:stream="http://etherx.jabber.org/streams"
    version="1.0">
  <pubsub xmlns="http://jabber.org/protocol/pubsub">
    <create node="/home/test2"/>
  </pubsub>
</iq>

订阅

Strophe.js代码

1
var subscribe = function (node) {
  var iq = $iq({
    type: 'set',
    from: jid,
    to: session.pubsub,
    id: getId()
  }).c('pubsub', {xmlns: 'http://jabber.org/protocol/pubsub'})
    .c('subscribe', {node: node, jid: jid});
  connection.send(iq.tree());
};

XML节

1
<!--客户端IQ-set请求-->
<iq type='set' from='root@xmpp.myserver.info'
    to='pubsub.xmpp.myserver.info' id='id-1412189193003'
    xmlns='jabber:client'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <subscribe node='/home/test' jid='root@xmpp.myserver.info'/>
  </pubsub>
</iq>
<!--服务端应答IQ-result-->
<iq from="pubsub.xmpp.myserver.info"
    to="root@xmpp.myserver.info/17735350171411959891145164"
    id="id-1412189193003" type="result"
    xmlns="jabber:client" xmlns:stream="http://etherx.jabber.org/streams"
    version="1.0">
  <pubsub xmlns="http://jabber.org/protocol/pubsub">
    <subscription
        jid="root@xmpp.myserver.info" subscription="subscribed" subid="583EAABFD09CF"
        node="/home/test"/>
  </pubsub>
</iq>

参考资料

  1. http://wiki.jabbercn.org/XEP-0060#.E5.88.9B.E5.BB.BA.E8.8A.82.E7.82.B9

XMPP XEP-0184消息回执

XEP-0184, 定义如下:

本规范定义了一个关于消息送达收条的XMPP协议扩展,消息的发送者可以要求消息的接收设备在接收到消息后发送一个确认消息(收条). 比如: 发邮件给对方可以要求一个邮件回执,以确定对方收到消息. 如果没有收到回执,可以重新发送.

应用程序简介

In OTP, application denotes a component implementing some specific functionality, that can be started and stopped as a unit, and which can be re-used in other systems as well. This module interfaces the application controller, a process started at every Erlang runtime system, and contains functions for controlling applications (for example starting and stopping applications), and functions to access information about applications (for example configuration parameters).

An application is defined by an application specification. The specification is normally located in an application resource file called Application.app, where Application is the name of the application. Refer to app(4) for more information about the application specification.

This module can also be viewed as a behaviour for an application implemented according to the OTP design principles as a supervision tree. The definition of how to start and stop the tree should be located in an application callback

在OPT中,一个应用程序表示一个实现了特定功能的组件, 可以作为独立的单元start和stop,可以在其他系统中重用.

application模块与application controller连接, application controller 是一个随erlang系统启动的进程.用于控制应用程序(启动,停止,访问应用程序信息).

  • application specification
    描述如何定义一个应用程序
  • application resource file(Application.app)
    Application为应用程序的名称
  • erl -man app
    查看应用程序资源文件的详细规范

Ejabberd模块 - 获取房间成员列表

上一篇Ejabberd 14.x版本系列模块开发过程中遇到的坑,一个很坑的移植过程,没找到相关的升级文档,全靠读代码. -_-!

本模块用于获取一个房间内的所有成员, 代码在这里, 本模块修改自获取房间的用户列表模块, 兼容ejabberd 14.x系列版本.

Bugfix

  1. muc_room表结构有变化, 下面是输出的日志,为了便于阅读,我在每个元素前附加序号作为前缀
1
[{muc_room,
    {<<"test">>,<<"conference.xmpp.myserver.info">>},
    [
        1  {title,<<232,129,138,229,164,169,229,174,164,230,160,135,233,162,152>>},
        2  {description,<<232,129,138,229,164,169,229,174,164,230,143,143,232,191,176>>},
        3  {allow_change_subj,true},
        4  {allow_query_users,true},
        5  {allow_private_messages,true},
        6  {allow_private_messages_from_visitors,anyone},
        7  {allow_visitor_status,true},
        8  {allow_visitor_nickchange,true},
        9  {public,true},
        10 {public_list,true},
        11 {persistent,true},
        12 {moderated,true},
        13 {members_by_default,true},
        14 {members_only,false},
        15 {allow_user_invites,false},
        16 {password_protected,false},
        17 {captcha_protected,false},
        18 {password,<<>>},
        19 {anonymous,true},
        20 {logging,false},
        21 {max_users,200},
        22 {allow_voice_requests,true},
        23 {voice_request_min_interval,1800},
        24 {vcard,<<>>},
        25 {captcha_whitelist,[]},
        26 {affiliations,[
            {
                {<<"hezhiqiang">>,<<"xmpp.myserver.info">>,<<>>},
                {owner,<<>>}
            }
        ]},
        27 {subject,<<>>},
        28 {subject_author,<<>>}
    ]}]

其中元素affiliations位于房间配置列表第26个元素,和原文<<获取房间的用户列表模块>>不同

补充

忘了一件事, 需要在$EJABBERD/tools/xmpp_codec.spec增加下列代码:

1
-xml(room_member_item,
     #elem{name = <<"item">>,
           xmlns = <<"http://jabber.org/protocol/muc#member-list">>,
           result = '$jid',
           attrs = [#attr{name = <<"jid">>,
                          required = true,
                          dec = {dec_jid, []},
                          enc = {enc_jid, []}}]}).
-xml(room_member,
     #elem{name = <<"query">>,
           xmlns = <<"http://jabber.org/protocol/muc#member-list">>,
           result = {room_member, '$items'},
           refs = [#ref{name = room_member_item,
                        label = '$items'}]}).

并执行make spec更新, 然后再 make && make install, 最后 ejabberdctl restart

参考资料

  1. http://blog.zlxstar.me/blog/2013/07/21/dicorvery-user-muclist
  2. http://www.ibm.com/developerworks/cn/opensource/os-erlang1
  3. Erlang标准库函数 lists:any/2
    http://www.erlang.org/doc/man/lists.html#any-2
  4. Erlang标准库函数 lists:nth/2
    http://www.erlang.org/doc/man/lists.html#nth-2

Ejabberd 14.x版本系列模块开发过程中遇到的坑

网上的资料多数都是2.x版本的, 从Ejabberd2.x系列迁移到14.x,路上遇到很多坑坑洼洼, 这里以一个mod_cputime模块为实例记录一下坑爹的过程.

自定义新的名称空间问题

如果你想要直接使用mod_disco:register_feature/2来创建一个名称空间,那是不行的,为什么不行, 下面我来细讲:

你会碰到各种badxml的错误. 从结果倒推,ejabberd应该是使用了xmpp_codec.erl来验证交互过程中传递的XML节的有效性,无效的统统抛弃,增强安全性,这样模块就不能随意增加名称空间.

设$EJABBERD_SRC为源码根目录.

  • 在14.07中, 首先需要在$EJABBERD_SRC/tools/xmpp_codec.spec增加你需要扩展的名称空间, 例如:
1
-xml(cpu_time,
     #elem{name = <<"time">>,
           xmlns = <<"ejabberd:cputime">>,
           result = '$cdata',
           cdata = #cdata{label = '$cdata', required = true }}).
-xml(cpu,
    #elem{name = <<"query">>,
          xmlns = <<"ejabberd:cputime">>,
          result = {cpu, '$time'},
          refs = [#ref{name = cpu_time,
                       label = '$time',
                       min = 0, max = 1}]}).

  • 然后在ejabberd的源代码根目录下执行make spec生成新的xmpp_codec.hrl,xmpp_codec.erl两个文件.

  • make && make install

  • ejabberdctl restart

还有一点要注意的时, 所有#xmlel, #jid, #iq这些记录中对应的XML标签名称, 属性名称, 以及CDATA的数据类型必须是类似<<"iq">>这种二进制字符串. 否则也会出现各种意想不到的错误.

Ejabberd的文档不多,也比较碎片化,为了避免掉进各种坑里面,还是多研究源码,多试错.

添加新名称空间iq处理模块

1
-module(mod_cputime).
-behaviour(gen_mod).
-export([
    start/2,
    stop/1,
    process_local_iq/3
]).
-include("ejabberd.hrl").
-include("jlib.hrl").
-include("logger.hrl").
-define(NS_CPUTIME, <<"ejabberd:cputime">>).
start(Host, Opts) ->
    IQDisc = gen_mod:get_opt(
        iqdisc,
        Opts,
        fun gen_iq_handler:check_type/1,
        one_queue
    ),
    mod_disco:register_feature(Host, ?NS_CPUTIME),
    gen_iq_handler:add_iq_handler(ejabberd_local, Host, ?NS_CPUTIME, ?MODULE, process_local_iq, IQDisc),
    ok.
stop(Host) ->
    mod_disco:unregister_feature(Host, ?NS_CPUTIME),
    gen_iq_handler:remove_iq_handler(ejabberd_local, Host, ?NS_CPUTIME),
    ok.
process_local_iq(_From, _To, #iq{type = Type, sub_el = SubEl} = IQ) ->
    case Type of
        set ->
            IQ#iq{type = error, sub_el = [SubEl, ?ERR_NOT_ALLOWED]};
        get ->
            CPUTime = element(1, erlang:statistics(runtime)) / 1000,
            SCPUTime = lists:flatten(io_lib:format("~.3f", [CPUTime])),
            Packet = #iq{type = result, sub_el = [
                #xmlel{name = <<"query">>, attrs = [{<<"xmlns">>, ?NS_CPUTIME}], children = [
                    #xmlel{name = <<"time">>, attrs = [], children = [{xmlcdata, list_to_binary(SCPUTime)}]}]}
            ]},
            Packet
    end.

把上面的代码放到$EJABBERD_SRC/src下面,然后make && make install,
在配置文件/etc/ejabberd/ejabberd.yml增加一个模块配置,如下:

1
##
## Modules enabled in all ejabberd virtual hosts.
##
modules:
  mod_cputime: {}

执行ejabberdctl restart重启ejabberd.

验证

客户端发起一个IQ-get请求

1
<iq type='get' to='xmpp.myserver.info' xmlns='jabber:client'>
  <query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>

服务端IQ-result应答

自定义名称空间的Features列表

验证模块是否能够工作

客户端发起IQ-get请求一个CPU时间

1
<iq type='get' to='xmpp.myserver.info' xmlns='jabber:client'>
  <query xmlns='ejabberd:cputime'/>
</iq>

服务端IQ-result应答

1
<iq from="xmpp.myserver.info"
    to="root@xmpp.myserver.info/1507629570141186050149354"
    type="result"
    xmlns="jabber:client"
    xmlns:stream="http://etherx.jabber.org/streams"
    version="1.0">
  <query xmlns="ejabberd:cputime">
    <time>3.840</time>
  </query>
</iq>

客户端代码在这里,包含三个文件, 下一篇把获取房间的用户列表模块移植到ejabberd 14.07

要用到二进制字符串的地方

这里强调一下

  • 定义名称空间的位置

    1
    define(NS_CPUTIME, <<"ejabberd:cputime">>).

  • XML元素的属性名称和属性值

  • jlib:jid_to_string()接受一个#jid或一个三元组,User,Server,Resource为字符串类型(Erlang中的字符串就是一个字符列表). 从jlib.erl中的定义我们可以看到,内部使用了iolist_to_binary对列表进行转换. 这里类型不要传错了
1
jid_to_string(#jid{user = User, server = Server,
		   resource = Resource}) ->
    jid_to_string({User, Server, Resource});
jid_to_string({N, S, R}) ->
    Node = iolist_to_binary(N),
    Server = iolist_to_binary(S),
    Resource = iolist_to_binary(R),
    S1 = case Node of
	   <<"">> -> <<"">>;
	   _ -> <<Node/binary, "@">>
	 end,
    S2 = <<S1/binary, Server/binary>>,
    S3 = case Resource of
	   <<"">> -> S2;
	   _ -> <<S2/binary, "/", Resource/binary>>
	 end,
    S3.

参考资料

  1. Ejabberd Developers Guide
    http://www.girlsnn.net/ejabberd/doc/dev.html#sec15
  2. 获取用户房间列表
    http://blog.zlxstar.me/blog/2013/07/21/dicorvery-user-muclist
  3. Strophe.js API文档
    http://strophe.im/strophejs/doc/1.1.3/files/strophe-js.html
  4. Ejabberd IQ 处理程序
    https://www.process-one.net/en/wiki/ejabberd_IQ_handlers/

Ejabberd中几个重要的Record结构

Ejabberd中几非常重要的记录结构, 贯穿整个Ejabberd的开发, 非常基础, 如果不能很好的理解,那么几乎不能很好的扩展Ejabberd的功能.

jid

Jabber 标识, 最基本的一个record结构, 定义在文件$EJABBERD_SRC/include/jlib.hrl中,包含6个字段

1
-record(jid, {user = <<"">> :: binary(),
              server = <<"">> :: binary(),
              resource = <<"">> :: binary(),
              luser = <<"">> :: binary(),
              lserver = <<"">> :: binary(),
              lresource = <<"">> :: binary()}).

iq

Info/Query 节, XMPP协议中三个基本的XML节之一, 并且是三个中最重要的一个, 大多数交互都是靠它来完成的. 定义在文件$EJABBERD_SRC/include/jlib.hrl中

1
-record(iq, {id = <<"">>       :: binary(),
             type = get        :: get | set | result | error,
             xmlns = <<"">>    :: binary(),
             lang  = <<"">>    :: binary(),
             sub_el = #xmlel{} :: xmlel() | [xmlel()]}).

xmlel

一个XML元素记录, 用于构造自定义XML节

Ejabberd-路由表

ejabberd内部模块可以通过一个XMPP名称(-define(PROCNAME, ejabberd_mod_echo).),添加自身到服务器的路由表中,这些模块被称为服务,服务模块必须同时实现gen_server和gen_mod行为

例子

mod_echo.erl是一个使用路由机制的例子.

API

1
ejabberd_router:register_route(Host)
ejabberd_router:unregister_route(Host),
* Host = string()

gen_server API

下面上个函数用作模块的API

1
start_link(Host, Opts) ->
    Proc = gen_mod:get_module_proc(Host, ?PROCNAME),
    gen_server:start_link({local, Proc}, ?MODULE, [Host, Opts], []).
1
start(Host, Opts) ->
    Proc = gen_mod:get_module_proc(Host, ?PROCNAME),
    ChildSpec = {Proc,{?MODULE, start_link, [Host, Opts]}, temporary, 1000, worker, [?MODULE]},
    supervisor:start_child(ejabberd_sup, ChildSpec).
1
stop(Host) ->
    Proc = gen_mod:get_module_proc(Host, ?PROCNAME),
    gen_server:call(Proc, stop),
    supervisor:terminate_child(ejabberd_sup, Proc),
    supervisor:delete_child(ejabberd_sup, Proc).

gen_server 回调

下面的函数必须定义并导出.

1
init([Host, Opts]) -> {ok, State} |
                      {ok, State, Timeout} |
                      ignore |
                      {stop, Reason}
1
handle_info(Info, State) -> {noreply, State} |
                            {noreply, State, Timeout} |
                            {stop, Reason, State}
* Info = {route, From, To, Packet}
* To = From = #jid (see jlib core module)
* Packet = {xmlelement, Name, Attrs, SubEl}
1
terminate(Reason, State) -> void()
1
handle_call(stop, _From, State) -> {stop, normal, ok, State}.
1
handle_cast(_Msg, State) -> {noreply, State}.
1
code_change(_OldVsn, State, _Extra) -> {ok, State}.

init函数用于初始化模块.Host为模块所在虚拟主机名称,Opts为在配置文件中设置的模块选项,这些选项可以通过gen_mod:get_opt/3函数获取.ejabberd_router:register_route/1函数在init回调中执行.

terminate/2用于停止模块. ejabberd_router:unregister_route函数在此回调中被调用.

handle_info/2用于获取发送给该模块的XMPP包. ejabberd_router:route/1用于对包进行重新路由.

All other callbacks can be written as shown above.

Ejabberd-IQ节处理程序

Ejabberd内部模块可以注册自身,并使用指定名称空间处理IQ节, 其方法类似于事件和钩子这种机制.

模块mod_last.erl就使用了这种机制, 同时也使用了事件和钩子

API

1
gen_iq_handler:add_iq_handler(Scope, Host, Namespace, Module, Function, IQDisc)
gen_iq_handler:remove_iq_handler(Scope, Host, Namespace, Module, Function, IQDisc)
* Scope = ejabberd_local | ejabberd_sm
* Namespace = string() (某些名称空间宏定义在jlib.hrl头文件中)
* Host = string()
* Module = atom()
* Fonction = atom()
* IQDisc = no_queue | one_queue | {queues, N} | parallel
* N = integer()

ejabberd_local作用域用于注册发到服务器本身的IQ.
ejabberd_sm作用域用于注册发送到一个账号的纯JID(Bare JID)的IQ,Host为与该IQ相关的虚拟主机名称.

Module和Function描述了一个处理器函数,当收到指定名称空间的IQ节时,该函数被调用.该处理器函数必须具有如下声明:

1
Module:Function(From, To, IQ) -> IQ
* From = To = #jid
* IQ = #iq

处理函数返回结果IQ (IQ-result), IQDisc描述当前IQ如何处理:

no_queue: 不创建新线程运行该处理程序
one_queue: 创建一个专用线程运行该处理程序
{queues, N}: 创建N个线程运行该处理程序
parallel: 对每一个接收到的IQ创建一个线程

IQ处理模块模板

修改以适应Ejabberd 14.07

1
-module(mod_iqtest).
-behaviour(gen_mod).
-export([start/2, stop/1,
  process_sm_iq/3 %% I assume this is needed to handle JID to JID communication of the IQ?
]).
-include("ejabberd.hrl").
-include("jlib.hrl").
-include("logger.hrl").
-define(NS_TEST, "http://jabber.org/protocol/test").
-define(NS_TEST2, "http://jabber.org/protocol/test2").
start(Host, Opts) ->
  IQDisc = gen_mod:get_opt(
    iqdisc,
    Opts,
    fun gen_iq_handler:check_type/1,
    one_queue
  ),
  gen_iq_handler:add_iq_handler(
    ejabberd_sm, Host, ?NS_TEST, ?MODULE, process_sm_iq, IQDisc
  ),
  gen_iq_handler:add_iq_handler(
    ejabberd_sm, Host, ?NS_TEST2, ?MODULE, process_sm_iq, IQDisc
  ),
  ?INFO_MSG("Loading module 'mod_iqtest' v.01", []).
stop(Host) ->
  gen_iq_handler:remove_iq_handler(
    ejabberd_sm, Host, ?NS_TEST
  ),
  gen_iq_handler:remove_iq_handler(
    ejabberd_sm, Host, ?NS_TEST2
  ),
  ?INFO_MSG("Stoping module 'mod_iqtest' ", []).
process_sm_iq(_From, _To, #iq{type = get, xmlns = ?NS_TEST} = IQ) ->
  ?INFO_MSG("Processing IQ Get query:~n ~p", [IQ]),
  IQ#iq{
    type = result, sub_el = [{xmlelement, "value", [], [{xmlcdata, "Hello World of Testing."}]}]
  };
process_sm_iq(_From, _To, #iq{type = get, xmlns = ?NS_TEST2} = IQ) ->
  ?INFO_MSG("Processing IQ Get query of namespace 2:~n ~p", [IQ]),
  IQ#iq{
    type = result, sub_el = [{xmlelement, "value", [], [{xmlcdata, "Hello World of Test 2."}]}]
  };
process_sm_iq(_From, _To, #iq{type = set} = IQ) ->
  ?INFO_MSG("Processing IQ Set: it does nothing", []),
  IQ#iq{
    type = result, sub_el = []
  };
process_sm_iq(_From, _To, #iq{sub_el = SubEl} = IQ) ->
  ?INFO_MSG("Processing IQ other type: it does nothing", []),
  IQ#iq{
    type = error, sub_el = [SubEl, ?ERR_FEATURE_NOT_IMPLEMENTED]
  }.

参考资料

  1. IQ处理程序模板
    https://www.ejabberd.im/node/5035?q=node/5035
  2. Ejabberd事件和钩子
    https://www.process-one.net/en/wiki/ejabberd_events_and_hooks
  3. mod_last模块
    https://github.com/processone/ejabberd/blob/master/src/mod_last.erl

Ejabberd中用Jiffy输出JSON数据

文章目录
  1. 1. 编译
  2. 2. 一个例子
  3. 3. 构造复杂JSON对象
    1. 3.1. Erlang和JSON格式对照表
    2. 3.2. 示例代码
    3. 3.3. 输出
  4. 4. 变更
    1. 4.1. 2014-10-04
  5. 5. 参考资料

警告! 使用Jiffy需要特别注意,其实现为NIF,可能导致Erlang VM崩溃.

编译

1
cd /tmp
git clone https://github.com/processone/ejabberd.git
cd ejabberd
chmod +x autogen.sh
./autogen.sh
./configure --enable-json --enable-mysql
make
make install

jiffy.so路径有个Bug,写作本文时,该BUG暂未解决. 请参考:
https://github.com/processone/ejabberd/issues/309,
要解决此问题, 执行:

1
mv /lib/ejabberd/priv/lib/jiffy.so /lib/ejabberd/priv/jiffy.so

一个例子

1
-module(mod_online_users).
-author('hezhiqiang').
-behaviour(gen_mod).
-export([
    start/2,
    stop/1,
    process/2
]).
-include("ejabberd.hrl").
-include("jlib.hrl").
-include("ejabberd_http.hrl").
-include("logger.hrl").
%% 处理函数,直接返回要输出到浏览器的内容
process(_LocalPath, _Request) ->
    Users = mnesia:table_info(session, size),
    OnlineUsers = jiffy:encode({[{<<"onlineusers">>, Users}]}),
    {200, [], OnlineUsers}.
start(_Host, _Opts) ->
    ?INFO_MSG("===Starting module mod_online_users===", []),
    ok.
stop(_Host) ->
    ?INFO_MSG("===Stopping module mod_online_users===", []),
    ok.

配置/etc/ejabberd/ejabberd.yml

1
-
  port: 5280
  module: ejabberd_http
  request_handlers:
    "/pub/archive": mod_http_fileserver
    "/http-bind/": mod_http_bind
    "admin": ejabberd_web_admin
    "online": mod_online_users
  web_admin: true
  http_poll: true
  http_bind: true
  captcha: true

重启

1
ejabberdctl restart

访问

1
http://192.168.8.132:5280/online/

在线用户数模块

构造复杂JSON对象

Erlang和JSON格式对照表

1
Erlang                          JSON            Erlang
==========================================================================
null                       -> null           -> null
true                       -> true           -> true
false                      -> false          -> false
"hi"                       -> [104, 105]     -> [104, 105]
<<"hi">>                   -> "hi"           -> <<"hi">>
hi                         -> "hi"           -> <<"hi">>
1                          -> 1              -> 1
1.25                       -> 1.25           -> 1.25
[]                         -> []             -> []
[true, 1.0]                -> [true, 1.0]    -> [true, 1.0]
{[]}                       -> {}             -> {[]}
{[{foo, bar}]}             -> {"foo": "bar"} -> {[{<<"foo">>, <<"bar">>}]}
{[{<<"foo">>, <<"bar">>}]} -> {"foo": "bar"} -> {[{<<"foo">>, <<"bar">>}]}
#{<<"foo">> => <<"bar">>}  -> {"foo": "bar"} -> #{<<"foo">> -> <<"bar">>}

示例代码

1
process(_LocalPath, _Request) ->
    ConnectedUsersNumber = ejabberd_sm:connected_users_number(),
    %% 获取在线用户列表
    AllSessionList = ejabberd_sm:dirty_get_sessions_list(),
    %% 构造JSON对象数组
    AllSessions = lists:map(fun({User, Server, Resource}) ->
        {[{user, User}, {server, Server}, {resource, Resource}]}
    end, AllSessionList),
    %% 编码JSON格式
    Json = jiffy:encode({[
        {connected_users_number, ConnectedUsersNumber},
        {sessions, AllSessions}
    ]}),
    {200, [], Json}.

输出

把上面的代码和下面的输出对照理解.

JSON输出

变更

2014-10-04

调用mnesia:system_info(all).获取Mnesia数据库信息, 下面是返回的Term:

1
[
    {access_module,mnesia},
    {auto_repair,true},
    {backup_module,mnesia_backup},
    {checkpoints,[]},
    {db_nodes,[ejabberd@localhost]},
    {debug,none},
    {directory,"/var/lib/ejabberd"},
    {dump_log_load_regulation,false},
    {dump_log_time_threshold,180000},
    {dump_log_update_in_place,true},
    {dump_log_write_threshold,1000},
    {event_module,mnesia_event},
    {extra_db_nodes,[]},
    {fallback_activated,false},
    {held_locks,[]},
    {ignore_fallback_at_startup,false},
    {fallback_error_function,{mnesia,lkill}},
    {is_running,yes},
    {local_tables,[
        shaper,
        mod_register_ip,local_config,caps_features,acl,
        access,carboncopy,http_bind,reg_users_counter,pubsub_subscription,bytestream,
        privacy,passwd,irc_custom,roster,last_activity,sr_user,roster_version,
        pubsub_last_item,offline_msg,route,motd,s2s,vcard,pubsub_index,sr_group,
        session_counter,vcard_search,motd_users,schema,session,private_storage,
        pubsub_item,muc_room,pubsub_state,iq_response,temporarily_blocked,
        muc_registered,muc_online_room,pubsub_node
    ]},
    {lock_queue,[]},
    {log_version,"4.3"},
    {master_node_tables,[]},
    {max_wait_for_decision,infinity},
    {protocol_version,{8,1}},
    {running_db_nodes,[ejabberd@localhost]},
    {schema_location,opt_disc},
    {schema_version,{2,0}},
    {subscribers,[<0.15778.0>,<0.16018.0>,<0.15961.0>,<0.15954.0>]},
    {tables,[
        carboncopy,http_bind,reg_users_counter,pubsub_subscription,bytestream,
        privacy,local_config,passwd,irc_custom,shaper,roster,last_activity,
        sr_user,roster_version,pubsub_last_item,offline_msg,route,motd,
        access,acl,s2s,vcard,pubsub_index,caps_features,sr_group,session_counter,
        mod_register_ip,vcard_search,motd_users,schema,session,private_storage,
        pubsub_item,muc_room,pubsub_state,iq_response,temporarily_blocked,
        muc_registered,muc_online_room,pubsub_node
    ]},
    {transaction_commits,56},
    {transaction_failures,81},
    {transaction_log_writes,0},
    {transaction_restarts,0},
    {transactions,[]},
    {use_dir,true},
    {core_dir,false},
    {no_table_loaders,2},
    {dc_dump_limit,4},
    {send_compressed,0},
    {version,"4.12.3"}
]

process/2函数修改为:

1
process(_LocalPath, _Request) ->
    ConnectedUsersNumber = ejabberd_sm:connected_users_number(),
    %% 用户列表
    AllSessionList = ejabberd_sm:dirty_get_sessions_list(),
    ?DEBUG("All sessions ~p~n", [AllSessionList]),
    %% Mnesia 表信息
    MnesiaSystemInfo = mnesia:system_info(all),
    ?DEBUG("Mnesia Information ~p~n", [MnesiaSystemInfo]),
    %% [{<<"root">>,<<"xmpp.myserver.info">>,<<"3439698832141213525690305">>}]
    %% 构造一个JSON对象数组
    %% 对象: {[]}
    %% 数组: []
    AllSessions = lists:map(fun({User, Server, Resource}) ->
        {[{user, User}, {server, Server}, {resource, Resource}]}
    end, AllSessionList),
    RemoveElements = [subscribers, fallback_error_function],
    MnesiaSystemInfoJiffy = lists:filtermap(fun({Key, Value}) ->
        case lists:any(fun(E2) -> E2 =:= Key end, RemoveElements) of
            true ->
                false;
            false ->
                case Key of
                    directory ->
                        {true, {Key, list_to_bitstring(Value)}};
                    version ->
                        {true, {Key, list_to_bitstring(Value)}};
                    log_version ->
                        {true, {Key, list_to_bitstring(Value)}};
                    schema_version ->
                        {V1, V2} = Value,
                        {true, {Key, list_to_bitstring([integer_to_list(V1), ".", integer_to_list(V2)])}};
                    protocol_version ->
                        {V1, V2} = Value,
                        {true, {Key, list_to_bitstring([integer_to_list(V1), ".", integer_to_list(V2)])}};
                    _ ->
                        {true, {Key, Value}}
                end
        end
    end, MnesiaSystemInfo),
    Json = jiffy:encode({[
        {connected_users_number, ConnectedUsersNumber},
        {sessions, AllSessions},
        {mnesia, {MnesiaSystemInfoJiffy}}
    ]}),
    {200, [], Json}.

JSON输出由JSON View Formater格式化
https://chrome.google.com/webstore/detail/hdmbdioamgdkppmocchpkjhbpfmpjiei

Mnesia系统信息JSON数据

注意lists:filtermap返回的是元组列表, 列表中的每个元素是一个元组, 每个元组包含两个项, lists:filtermap函数是lists:filter(过滤)和lists:map(映射)两个函数功能上的合并:过滤并映射

这里例子中有部分值为空[], 比如checkpoints, 当包含值的时候可能需要按照Jiffy方式转换类型.

完整代码如下:
https://gist.github.com/553129e3015995b56028

参考资料

  1. https://github.com/davisp/jiffy
  2. http://www.erlang.org/doc/man/lists.html
  3. http://www.erlang.org/doc/man/mnesia.html

Ejabberd C2S模块状态分析

Ejabberd的ejabberd_c2s核心模块,是处理XMPP协议的核心处理模块,本文通过客户端和服务器的交互过程来分析其原理.

建立TCP连接

客户端建立TCP连接ejabberd_listener接受TCP连接,服务器日志输出

1
# ejabberd_listener 接收到客户端的TCP连接请求
2014-09-26 05:22:31.527 [info] <0.673.0>@ejabberd_listener:accept:313 (#Port<0.5846>) Accepted connection 172.17.42.1:53457 -> 172.17.0.27:5222

ejabberd_c2s进程初始状态为wait_for_stream,等待接收{xmlstreamstart, _Name, Attrs}消息

客户端打开流

客户端发送打开流Stanza

1
<stream:stream to='xmpp.myserver.info' xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams' version='1.0'>

服务器日志输出为

1
# ejabberd_receiver 接收到客户端的流打开请求
2014-09-26 05:22:31.527 [debug] <0.849.0>@ejabberd_receiver:process_data:343 Received XML on stream = <<"<stream:stream xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client" to="xmpp.myserver.info" version="1.0">">>
# 服务器确认客户的流打开请求,返回一个响应
2014-09-26 05:22:31.528 [debug] <0.850.0>@ejabberd_c2s:send_text:1869 Send XML on stream = <<"<?xml version='1.0'?><stream:stream xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams' id='3177127870' from='xmpp.myserver.info' version='1.0' xml:lang='en'>">>
# 服务器返回功能相应
2014-09-26 05:22:31.529 [debug] <0.850.0>@ejabberd_c2s:send_text:1869 Send XML on stream = <<"<stream:features><mechanisms xmlns='urn:ietf:params:xml:ns:xmpp-sasl'><mechanism>DIGEST-MD5</mechanism><mechanism>SCRAM-SHA-1</mechanism><mechanism>PLAIN</mechanism></mechanisms><c xmlns='http://jabber.org/protocol/caps' hash='sha-1' node='http://www.process-one.net/en/ejabberd/' ver='aIT+/ulfcbHXDKPkCA+iw9x5mU8='/><register xmlns='http://jabber.org/features/iq-register'/></stream:features>">>

ejabberd_c2s状态更新为wait_for_feature_request,等待接收{xmlstreamstart, _Name, Attrs}消息

客户端认证

客户端发送一个<auth>请求认证

1
# auth内的文本值无换行和空格,这里为了可读性而格式化
<auth xmlns='urn:ietf:params:xml:ns:xmpp-sasl' mechanism='SCRAM-SHA-1'>
    biwsbj1yb290LHI9ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U=
</auth>

服务器收到认证请求 并更新内部状态

1
# 客户端请求认证
2014-09-26 05:22:31.543 [debug] <0.849.0>@ejabberd_receiver:process_data:343 Received XML on stream = <<"<auth xmlns="urn:ietf:params:xml:ns:xmpp-sasl" mechanism="SCRAM-SHA-1">biwsbj1yb290LHI9ZDQxZDhjZDk4ZjAwYjIwNGU5ODAwOTk4ZWNmODQyN2U=</auth>">>
# 更新内部状态
2014-09-26 05:22:31.543 [debug] <0.849.0>@shaper:update:117 State: {maxrate,1000,0.0,1411708951528734}, Size=138

此时, ejabberd_c2s状态更新为wait_for_sasl_response, 同时返回<challenge>响应

1
# challenge内的文本值无换行和空格,这里为了可读性而格式化
<challenge
    xmlns="urn:ietf:params:xml:ns:xmpp-sasl"
    xmlns:stream="http://etherx.jabber.org/streams" version="1.0">
    cj1kNDFkOGNkOThmMDBiMjA0ZTk4MDA5...省略</challenge>

客户端响应<challenge>

1
<response
    xmlns='urn:ietf:params:xml:ns:xmpp-sasl'>
    Yz1iaXdzLHI9ZDQxZDhjZDk4ZjAw...省略</response>

服务器返回<success>

1
<success
    xmlns="urn:ietf:params:xml:ns:xmpp-sasl"
    xmlns:stream="http://etherx.jabber.org/streams"
    version="1.0">dj1saHU0SHBCZVVsc2Fla2dhUFN2cXlxZ3Jxdlk9</success>

ejabberd_c2s状态重新回到wait_for_stream,内部状态StateData#state.authenticated为非false

绑定资源

客户端重新发送<stream>

1
<stream:stream
    to='xmpp.myserver.info'
    xmlns='jabber:client'
    xmlns:stream='http://etherx.jabber.org/streams'
    version='1.0'>

服务器确认,并发送功能节

1
<stream:stream xmlns="jabber:client"
    xmlns:stream="http://etherx.jabber.org/streams"
    id="111324954"
    from="xmpp.myserver.info"
    version="1.0"
    xml:lang="en">
1
<stream:features xmlns="jabber:client"
    xmlns:stream="http://etherx.jabber.org/streams" version="1.0">
  <bind xmlns="urn:ietf:params:xml:ns:xmpp-bind"/>
  <session xmlns="urn:ietf:params:xml:ns:xmpp-session"/>
  <sm xmlns="urn:xmpp:sm:2"/>
  <sm xmlns="urn:xmpp:sm:3"/>
  <csi xmlns="urn:xmpp:csi:0"/>
  <c xmlns="http://jabber.org/protocol/caps"
    hash="sha-1"
    node="http://www.process-one.net/en/ejabberd/"
    ver="aIT+/ulfcbHXDKPkCA+iw9x5mU8="/>
  <register xmlns="http://jabber.org/features/iq-register"/>
</stream:features>

客户端选择绑定资源(资源由服务器生成)

1
<iq type='set' id='_bind_auth_2' xmlns='jabber:client'>
  <bind xmlns='urn:ietf:params:xml:ns:xmpp-bind'/>
</iq>

服务器应答

1
<iq id="_bind_auth_2"
    type="result"
    xmlns="jabber:client"
    xmlns:stream="http://etherx.jabber.org/streams"
    version="1.0">
  <bind xmlns="urn:ietf:params:xml:ns:xmpp-bind">
    <jid>root@xmpp.myserver.info/39189873701411708951913434</jid>
  </bind>
</iq>

客户端请求初始化会话

1
<iq type='set' id='_session_auth_2' xmlns='jabber:client'>
  <session xmlns='urn:ietf:params:xml:ns:xmpp-session'/>
</iq>

服务器应答

1
<iq type="result"
    xmlns="jabber:client"
    id="_session_auth_2"
    xmlns:stream="http://etherx.jabber.org/streams"
    version="1.0"/>

参考资料

  1. 发现一篇类似的分析博客
    http://my.oschina.net/hncscwc/blog/159826

Ejabberd模块快速入门

Ejabberd是一个以Elang编程语言开发的开源XMPP服务器.过去很长时间,XMPP以构建即时通信应用程序闻名, 很多人用它来构建实时应用程序. 依赖预Erlang平台的并发特性,在选择XMPP服务器的时候,Ejabberd天生的就适合与处理大规模并发连接的应用环境.

Erlang 尾递归

Elrang尾递归和PHP的for循环

1
for(0)->
        ok;
for(N)->
    io:format("running time: ~p ms ~n",
        [merle:getkey("test")]),
    for(N-1).

另一种方式,采用列表,通过模式匹配[Head|Tail]的形式

编译Ejabberd遇到的问题

问题: 缺少libyaml库

办法: 安装需要的库

Ejabberd 从 13.10 开始配置文件的格式从Erlang Term转换到使用YAML格式, 需要用到libyaml来解析yaml文件.

浏览器打开: http://pyyaml.org/download/libyaml/, 找到最新版本的libyaml

1
wget http://pyyaml.org/download/libyaml/yaml-0.1.6.tar.gz
tar zxf yaml-0.1.6.tar.gz
./configure
make
sudo make install

问题: 网络中断导致下载了不完整的deps/包, 比如deps/p1_yaml

办法: 删除不完整的包,重新make

1
rm -rf deps/p1_yaml
make

问题: 编译ejabberd模块找不到头文件的问题

办法: 网上很多介绍编译模块的方法是使用erlc编译模块, 使用erlc编译模块需要指明头文件,依赖库的路径,容易出错,编译自定义的ejabberd并没有这么麻烦,只要把写好了的ejabberd模块文件放到$(EJABBERD)/src目录下, 然后运行make, 生成的.beam文件会自动生成到`$(EJABBERD)/ebin目录下.