请选择 进入手机版 | 继续访问电脑版

Redis中国用户组(CRUG)论坛

 找回密码
 立即注册

QQ登录

只需一步,快速开始

扫一扫,访问微社区

搜索
热搜: 活动 交友 discuz
查看: 2661|回复: 0

Redis源码分析(三十一)--- latency延迟分析处理

[复制链接]
  • TA的每日心情
    开心
    2017-3-20 10:39
  • 签到天数: 90 天

    [LV.6]常住居民II

    358

    主题

    462

    帖子

    3650

    积分

    管理员

    Rank: 9Rank: 9Rank: 9

    积分
    3650

    最佳新人活跃会员宣传达人突出贡献优秀版主荣誉管理论坛元老

    发表于 2016-4-14 10:03:09 | 显示全部楼层 |阅读模式

    每当提到延时统计的时候,一定想到的一个名词就是”性能测试“,没错,在Redis的redis_benchmark文件中,的确用到了延迟文件中的相关信息。在Redis中的官方解释此文件:

    1. /* The latency monitor allows to easily observe the sources of latency  
    2. * in a Redis instance using the LATENCY command. Different latency  
    3. * sources are monitored, like disk I/O, execution of commands, fork  
    4. * system call, and so forth.  
    5. *  
    6. * 延时监听器可以对Redis中很多简单的资源进行监听,比如I/O磁盘操作,执行一些指令,  
    7. * fork创建子线程操作等的监听。  
    8. * ----------------------------------------------------------------------------  
    复制代码


    在Redis中的延时操作中,整个过程原理非常简单,他是针对每种事件维护了一个统计列表,每个列表中包括了了采集的一系列样本,每个样本包括,此样本的创建时间和此样本的延时时间。event==》对SampleSeriesList 是一个字典的映射关系。下面看看,里面关键的采集点,名叫latencySample采集点的结构定义:

    1. /* Representation of a latency sample: the sampling time and the latency
    2. * observed in milliseconds. */  
    3. /* 延时样品例子 */  
    4. struct latencySample {  
    5.     //延时Sample创建的时间  
    6.     int32_t time; /* We don't use time_t to force 4 bytes usage everywhere. */  
    7.     //延时的具体时间, 单位为毫秒  
    8.     uint32_t latency; /* Latency in milliseconds. */  
    9. };  
    复制代码

    字典中维护的可不是一个Sample结点,而是一个结点列表结构体:
    1. /* The latency time series for a given event. */  
    2. /* 针对某个事件采集的一系列延时sample */  
    3. struct latencyTimeSeries {  
    4.     //下一个延时Sample的下标  
    5.     int idx; /* Index of the next sample to store. */  
    6.     //最大的延时  
    7.     uint32_t max; /* Max latency observed for this event. */  
    8.     //最近的延时记录  
    9.     struct latencySample samples[LATENCY_TS_LEN]; /* Latest history. */  
    10. };  
    复制代码

    在Redis代码的设计中,因为延时是用来测试和结果分析的,所以,作者还设计了用于后面分析报告中会用到的数据统计结构体;
    1. /* Latency statistics structure. */  
    2. /* 延时sample的数据统计结果结构体 */  
    3. struct latencyStats {  
    4.     //绝对最高的延时时间  
    5.     uint32_t all_time_high; /* Absolute max observed since latest reset. */  
    6.     //平均Sample延时时间  
    7.     uint32_t avg;           /* Average of current samples. */  
    8.     //Sample的最小延时时间  
    9.     uint32_t min;           /* Min of current samples. */  
    10.     //Sample的最大延时时间  
    11.     uint32_t max;           /* Max of current samples. */  
    12.     //平均相对误差,与平均延时相比  
    13.     uint32_t mad;           /* Mean absolute deviation. */  
    14.     //samples的总数  
    15.     uint32_t samples;       /* Number of non-zero samples. */  
    16.     //最早的延时记录点的创建时间  
    17.     time_t period;          /* Number of seconds since first event and now. */  
    18. };  
    复制代码

       意思都非常的直接,那么一个简单的Sample如何进行事件的检测呢?
    1. /* Start monitoring an event. We just set the current time. */  
    2. /* 对某个事件设置监听,就是设置一下当前的时间 */  
    3. #define latencyStartMonitor(var) if (server.latency_monitor_threshold) { \  
    4.     var = mstime(); \  
    5. } else { \  
    6.     var = 0; \  
    7. }  
    8.   
    9. /* End monitoring an event, compute the difference with the current time
    10. * to check the amount of time elapsed. */  
    11. /* 结束监听,算出过了多少时间 */  
    12. #define latencyEndMonitor(var) if (server.latency_monitor_threshold) { \  
    13.     var = mstime() - var; \  
    14. }  
    复制代码

    很简单,记录开始时间,记录结束时间,中间的差值就是延时时间了,如果超出给定的时间范围,就加入到延时列表中:
    1. /* Add the sample only if the elapsed time is >= to the configured threshold. */  
    2. /* 如果延时时间超出server.latency_monitor_threshold,则将Sample加入延时列表中 */  
    3. #define latencyAddSampleIfNeeded(event,var) \  
    4.     if (server.latency_monitor_threshold && \  
    5.         (var) >= server.latency_monitor_threshold) \  
    6.           latencyAddSample((event),(var));  
    复制代码

    我们重点关注一下,latencyAddSample,就是把采样结点加入到记录中,步骤如下:

    1.根据传入的event事件,在server.latency_events找到key为event事件 的val,即一个latencyTimeSeries

    2.在这个latencyTimeSeries的struct latencySample samples[LATENCY_TS_LEN]中添加一个新的Sample

    实现代码如下:

    1. /* Add the specified sample to the specified time series "event".
    2. * This function is usually called via latencyAddSampleIfNeeded(), that
    3. * is a macro that only adds the sample if the latency is higher than
    4. * server.latency_monitor_threshold. */  
    5. /* 添加Sample到指定的Event对象的Sample列表中 */  
    6. void latencyAddSample(char *event, mstime_t latency) {  
    7.     //找出Event对应的延时Sample记录结构体  
    8.     struct latencyTimeSeries *ts = dictFetchValue(server.latency_events,event);  
    9.     time_t now = time(NULL);  
    10.     int prev;  
    11.   
    12.     /* Create the time series if it does not exist. */  
    13.     if (ts == NULL) {  
    14.         ts = zmalloc(sizeof(*ts));  
    15.         ts->idx = 0;  
    16.         ts->max = 0;  
    17.         memset(ts->samples,0,sizeof(ts->samples));  
    18.         //如果ts为空,重新添加,一个Event,对应一个latencyTimeSeries  
    19.         dictAdd(server.latency_events,zstrdup(event),ts);  
    20.     }  
    21.   
    22.     /* If the previous sample is in the same second, we update our old sample
    23.      * if this latency is > of the old one, or just return. */  
    24.     prev = (ts->idx + LATENCY_TS_LEN - 1) % LATENCY_TS_LEN;  
    25.     if (ts->samples[prev].time == now) {  
    26.         if (latency > ts->samples[prev].latency)  
    27.             ts->samples[prev].latency = latency;  
    28.         return;  
    29.     }  
    30.   
    31.     //为Sample赋值  
    32.     ts->samples[ts->idx].time = time(NULL);  
    33.     ts->samples[ts->idx].latency = latency;  
    34.     if (latency > ts->max) ts->max = latency;  
    35.   
    36.     ts->idx++;  
    37.     if (ts->idx == LATENCY_TS_LEN) ts->idx = 0;  
    38. }  
    复制代码

    结点都出来之后,当然会进行结构的分析统计了,这时就用到了latencyStats结构体;
    1. /* Analyze the samples avaialble for a given event and return a structure
    2. * populate with different metrics, average, MAD, min, max, and so forth.
    3. * Check latency.h definition of struct latenctStat for more info.
    4. * If the specified event has no elements the structure is populate with
    5. * zero values. */  
    6. /* 分析某个时间Event的延时结果,结果信息存入latencyStats结构体中 */  
    7. void analyzeLatencyForEvent(char *event, struct latencyStats *ls) {  
    8.     struct latencyTimeSeries *ts = dictFetchValue(server.latency_events,event);  
    9.     int j;  
    10.     uint64_t sum;  
    11.       
    12.     //初始化延时统计结果结构体的变量  
    13.     ls->all_time_high = ts ? ts->max : 0;  
    14.     ls->avg = 0;  
    15.     ls->min = 0;  
    16.     ls->max = 0;  
    17.     ls->mad = 0;  
    18.     ls->samples = 0;  
    19.     ls->period = 0;  
    20.     if (!ts) return;  
    21.   
    22.     /* First pass, populate everything but the MAD. */  
    23.     sum = 0;  
    24.     for (j = 0; j < LATENCY_TS_LEN; j++) {  
    25.         if (ts->samples[j].time == 0) continue;  
    26.         ls->samples++;  
    27.         if (ls->samples == 1) {  
    28.             ls->min = ls->max = ts->samples[j].latency;  
    29.         } else {  
    30.             //找出延时最大和最小的延时时间  
    31.             if (ls->min > ts->samples[j].latency)  
    32.                 ls->min = ts->samples[j].latency;  
    33.             if (ls->max < ts->samples[j].latency)  
    34.                 ls->max = ts->samples[j].latency;  
    35.         }  
    36.         sum += ts->samples[j].latency;  
    37.   
    38.         /* Track the oldest event time in ls->period. */  
    39.         if (ls->period == 0 || ts->samples[j].time < ls->period)  
    40.             //最早的延时记录点的创建时间  
    41.             ls->period = ts->samples[j].time;  
    42.     }  
    43.   
    44.     /* So far avg is actually the sum of the latencies, and period is
    45.      * the oldest event time. We need to make the first an average and
    46.      * the second a range of seconds. */  
    47.     if (ls->samples) {  
    48.         ls->avg = sum / ls->samples;  
    49.         ls->period = time(NULL) - ls->period;  
    50.         if (ls->period == 0) ls->period = 1;  
    51.     }  
    52.   
    53.     /* Second pass, compute MAD. */  
    54.     //计算平均相对误差,与平均延时相比  
    55.     sum = 0;  
    56.     for (j = 0; j < LATENCY_TS_LEN; j++) {  
    57.         int64_t delta;  
    58.   
    59.         if (ts->samples[j].time == 0) continue;  
    60.         delta = (int64_t)ls->avg - ts->samples[j].latency;  
    61.         if (delta < 0) delta = -delta;  
    62.         sum += delta;  
    63.     }  
    64.     if (ls->samples) ls->mad = sum / ls->samples;  
    65. }  
    复制代码

    当然还可以利用这些采集的点,画一个微线图,更加形象的展示出来:
    1. #define LATENCY_GRAPH_COLS 80  
    2. /* 利用延时的Sample点,画出对应的微线图 */  
    3. sds latencyCommandGenSparkeline(char *event, struct latencyTimeSeries *ts) {  
    4.     int j;  
    5.     struct sequence *seq = createSparklineSequence();  
    6.     sds graph = sdsempty();  
    7.     uint32_t min = 0, max = 0;  
    8.   
    9.     for (j = 0; j < LATENCY_TS_LEN; j++) {  
    10.         int i = (ts->idx + j) % LATENCY_TS_LEN;  
    11.         int elapsed;  
    12.         char *label;  
    13.         char buf[64];  
    14.   
    15.         if (ts->samples[i].time == 0) continue;  
    16.         /* Update min and max. */  
    17.         if (seq->length == 0) {  
    18.             min = max = ts->samples[i].latency;  
    19.         } else {  
    20.             if (ts->samples[i].latency > max) max = ts->samples[i].latency;  
    21.             if (ts->samples[i].latency < min) min = ts->samples[i].latency;  
    22.         }  
    23.         /* Use as label the number of seconds / minutes / hours / days
    24.          * ago the event happened. */  
    25.         elapsed = time(NULL) - ts->samples[i].time;  
    26.         if (elapsed < 60)  
    27.             snprintf(buf,sizeof(buf),"%ds",elapsed);  
    28.         else if (elapsed < 3600)  
    29.             snprintf(buf,sizeof(buf),"%dm",elapsed/60);  
    30.         else if (elapsed < 3600*24)  
    31.             snprintf(buf,sizeof(buf),"%dh",elapsed/3600);  
    32.         else  
    33.             snprintf(buf,sizeof(buf),"%dd",elapsed/(3600*24));  
    34.         label = zstrdup(buf);  
    35.         sparklineSequenceAddSample(seq,ts->samples[i].latency,label);  
    36.     }  
    37.   
    38.     graph = sdscatprintf(graph,  
    39.         "%s - high %lu ms, low %lu ms (all time high %lu ms)\n", event,  
    40.         (unsigned long) max, (unsigned long) min, (unsigned long) ts->max);  
    41.     for (j = 0; j < LATENCY_GRAPH_COLS; j++)  
    42.         graph = sdscatlen(graph,"-",1);  
    43.     graph = sdscatlen(graph,"\n",1);  
    44.     //调用sparkline函数画微线图  
    45.     graph = sparklineRender(graph,seq,LATENCY_GRAPH_COLS,4,SPARKLINE_FILL);  
    46.     freeSparklineSequence(seq);  
    47.     //返回微线图字符串  
    48.     return graph;  
    49. }  
    复制代码

    在Redis还封装了一些命令供外部调用,这里就不分析了,就是对上述方法的复合调用:
    1. /* ---------------------------- Latency API --------------------------------- */  
    2. void latencyMonitorInit(void) /* 延时监听初始化操作,创建Event字典对象 */  
    3. void latencyAddSample(char *event, mstime_t latency) /* 添加Sample到指定的Event对象的Sample列表中 */  
    4. int latencyResetEvent(char *event_to_reset) /* 重置Event事件的延迟,删除字典中的event的记录 */  
    5. void analyzeLatencyForEvent(char *event, struct latencyStats *ls) /* 分析某个时间Event的延时结果,结果信息存入latencyStats结构体中 */  
    6. sds createLatencyReport(void) /* 根据延时Sample的结果,创建阅读性比较好的分析报告 */  
    7. void latencyCommandReplyWithSamples(redisClient *c, struct latencyTimeSeries *ts)  
    8. void latencyCommandReplyWithLatestEvents(redisClient *c)  
    9. sds latencyCommandGenSparkeline(char *event, struct latencyTimeSeries *ts)  
    10. void latencyCommand(redisClient *c)  
    复制代码

    Redis的延时类文件的分析也结束了,分析了这么长时间Redis的Redis代码,感觉每一块的代码都会有他的亮点存在,分析了30多期下来,还是学到了很多网上所学不到的知识,网上更多的是Redis主流思想的学习,像一些比较细小点,也只有自己品味,自己才能够真正的体会。


    转自:http://blog.csdn.net/androidlushangderen/article/details/40818553
    上一篇:Redis源码分析(三十)--- pubsub发布订阅模式
    下一篇:Redis源码分析(三十二)--- redis-cli客户端命令行接口的实现



    您需要登录后才可以回帖 登录 | 立即注册

    本版积分规则

    阿里云
    阿里云

    Archiver|手机版|小黑屋|Redis中国用户组 ( 京ICP备15003959号

    GMT+8, 2017-3-29 15:19 , Processed in 0.109719 second(s), 32 queries .

    Powered by Discuz! X3.2

    © 2001-2013 Comsenz Inc.

    快速回复 返回顶部 返回列表