王筝的博客
ruby学习
# cat /etc/issue
Debian GNU/Linux 8 \n \l

#apt-get update

 

10:49 [webuser@a02]$ du -hs *
45M    m-video-fetcher-for-y
758M    tv-c-cms
118M    tv-y-cms
162M    virtual-r-cms
tty:[4] jobs:[0] cwd:[/opt/app/ruby]
10:49 [webuser@a02]$ du -hs m-cms/*
0    m-cms/current
721M    m-cms/releases
20M    m-cms/shared
tty:[4] jobs:[0] cwd:[/opt/app/ruby]
10:50 [webuser@a02]$ du -hs m-cms/releases/*
80M    m-cms/releases/20150410072949
80M    m-cms/releases/20150410073937
80M    m-cms/releases/20150414065535
80M    m-cms/releases/20150414070107
80M    m-cms/releases/20150423034731
81M    m-cms/releases/20150511032009
81M    m-cms/releases/20150513062949
81M    m-cms/releases/20150617092032
81M    m-cms/releases/20150618083013
tty:[4] jobs:[0] cwd:[/opt/app/ruby]
10:50 [webuser@a02.test.stat.3g.vm.m6.youku]$ ll m-cms/releases/20150410072949/
总用量 88
drwxrwxrwx 8 1000 1000 4096 4月  10 2015 app
-rwxrwxrwx 1 1000 1000  655 4月  10 2015 backup_log_files.rb
-rwxrwxrwx 1 1000 1000  269 4月  10 2015 Capfile
drwxrwxrwx 5 1000 1000 4096 4月  10 2015 config
-rwxrwxrwx 1 1000 1000  153 4月  10 2015 config.ru
drwxrwxrwx 3 1000 1000 4096 4月  10 2015 db
-rwxrwxrwx 1 1000 1000 1263 4月  10 2015 Gemfile
-rwxrwxrwx 1 1000 1000 5679 4月  10 2015 Gemfile.lock
-rwxrwxrwx 1 root root    0 4月  10 2015 httparty.log
drwxrwxrwx 4 1000 1000 4096 4月  10 2015 lib
lrwxrwxrwx 1 root root   37 4月  10 2015 log -> /opt/app/ruby/m-cms/shared/log
drwxrwxrwx 5 1000 1000 4096 4月  10 2015 operations
drwxrwxrwx 7 1000 1000 4096 4月  10 2015 public
-rwxrwxrwx 1 1000 1000  281 4月  10 2015 Rakefile
-rwxrwxrwx 1 1000 1000    1 4月  10 2015 REVISION
-rwxrwxrwx 1 1000 1000   26 4月  10 2015 robots.txt
drwxrwxrwx 2 1000 1000 4096 4月  10 2015 script
drwxrwxrwx 7 1000 1000 4096 4月  10 2015 spec
-rwxrwxrwx 1 1000 1000  630 4月  10 2015 start_tasks
-rwxrwxrwx 1 1000 1000  132 4月  10 2015 test_basic.rb
-rwxrwxrwx 1 1000 1000   80 4月  10 2015 test_map.rb
drwxrwxrwx 5 1000 1000 4096 4月  10 2015 tmp
drwxrwxrwx 3 1000 1000 4096 4月  10 2015 vendor
tty:[4] jobs:[0] cwd:[/opt/app/ruby]
10:50 [webuser@a02]$ ll -a -S m-cms/releases/20150410072949/
总用量 112
-rwxrwxrwx  1 1000 1000 5679 4月  10 2015 Gemfile.lock
drwxrwxrwx 14 1000 1000 4096 4月  10 2015 .
drwxrwxrwx 11 root root 4096 6月  18 2015 ..
drwxrwxrwx  8 1000 1000 4096 4月  10 2015 app
drwxrwxrwx  5 1000 1000 4096 4月  10 2015 config
drwxrwxrwx  3 1000 1000 4096 4月  10 2015 db
drwxrwxrwx  8 1000 1000 4096 4月  10 2015 .git
drwxrwxrwx  4 1000 1000 4096 4月  10 2015 .idea
drwxrwxrwx  4 1000 1000 4096 4月  10 2015 lib
drwxrwxrwx  5 1000 1000 4096 4月  10 2015 operations
drwxrwxrwx  7 1000 1000 4096 4月  10 2015 public
drwxrwxrwx  2 1000 1000 4096 4月  10 2015 script
drwxrwxrwx  7 1000 1000 4096 4月  10 2015 spec
drwxrwxrwx  5 1000 1000 4096 4月  10 2015 tmp
drwxrwxrwx  3 1000 1000 4096 4月  10 2015 vendor
-rwxrwxrwx  1 1000 1000 1263 4月  10 2015 Gemfile
-rwxrwxrwx  1 1000 1000  655 4月  10 2015 backup_log_files.rb
-rwxrwxrwx  1 1000 1000  630 4月  10 2015 start_tasks
-rwxrwxrwx  1 1000 1000  606 4月  10 2015 .gitignore
-rwxrwxrwx  1 1000 1000  281 4月  10 2015 Rakefile
-rwxrwxrwx  1 1000 1000  269 4月  10 2015 Capfile
-rwxrwxrwx  1 1000 1000  153 4月  10 2015 config.ru
-rwxrwxrwx  1 1000 1000  132 4月  10 2015 test_basic.rb
-rwxrwxrwx  1 1000 1000   80 4月  10 2015 test_map.rb
lrwxrwxrwx  1 root root   37 4月  10 2015 log -> /opt/app/ruby/m-cms/shared/log
-rwxrwxrwx  1 1000 1000   26 4月  10 2015 robots.txt
-rwxrwxrwx  1 1000 1000    9 4月  10 2015 .rspec
-rwxrwxrwx  1 1000 1000    1 4月  10 2015 REVISION
-rwxrwxrwx  1 root root    0 4月  10 2015 httparty.log
tty:[4] jobs:[0] cwd:[/opt/app/ruby]
10:50 [webuser@a02]$ du -hs m-cms/releases/20150410072949/.git
67M    m-cms/releases/20150410072949/.git

 

https://github.com/rbenv/rbenv#installation

安装ruby的第三步

Run ~/.rbenv/bin/rbenv init for shell-specific instructions on how to initialize rbenv to enable shims and autocompletion.

 

[webuser@iZ94aiq7f6cZ history-book]$ ~/.rbenv/bin/rbenv init
# Load rbenv automatically by appending
# the following to ~/.bash_profile:

eval “$(rbenv init -)”

注意:这句话需要手动添加到~/.bash_profile中,用指令执行后只在当前窗口生效

[webuser@iZ94aiq7f6cZ ~]$ ruby -v
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-linux]
[webuser@iZ94aiq7f6cZ ~]$ rbenv versions
system
* 2.3.0 (set by /home/webuser/.rbenv/version)

chmod +x analyse_log copy_log

 

(1)拷贝脚本到本地

脚本内容如下:

#!/usr/bin/env bash

LOG_PATH=/opt/logs
LOCAL_LOG_PATH=~/Documents/logs/8-15
PROJECTS=(m-cms-for-tudou-tv m-cms-for-tv)

if [[ $1 == "--debug" ]]; then
  set -x
fi

# Create local directories
for project in ${projects[@]}; do
  mkdir -p $SAVE_TO/$project
done

# Copy m-cms-for-tv
project=m-cms-for-tv
remote_log_dir=$LOG_PATH/$project

for server in {13..15}; do
  for port in {5200..5207}; do
    server_ip="10.xxx.32.$server"
    local_log_dir=$LOCAL_LOG_PATH/$project/$server_ip
    log_file="thin server (0.0.0.0:$port)_2015-08-15.log"

    if [[ ! -e $local_log_dir/"$log_file" ]]; then
      mkdir -p $local_log_dir
      scp $server_ip:$remote_log_dir/"'$log_file'" $local_log_dir/$port.log
    fi
  done
done

# Copy m-cms-for-todou-tv
project=m-cms-for-tudou-tv
remote_log_dir=$LOG_PATH/$project

for server in {13..15}; do
  for port in {3201..3208}; do
    server_ip="10.xxx.32.$server"
    local_log_dir=$LOCAL_LOG_PATH/$project/$server_ip
    log_file="thin server (0.0.0.0:$port)_2015-08-15.log"

    if [[ ! -e $local_log_dir/"$log_file" ]]; then
      mkdir -p $local_log_dir
      scp $server_ip:$remote_log_dir/"'$log_file'" $local_log_dir/$port.log
    fi
  done
done

project=ott-client-cms
remote_log_dir=/opt/app/ruby/$project/current/log

for server in {13..15}; do
  server_ip="10.xxx.32.$server"
  local_log_dir=$LOCAL_LOG_PATH/$project/$server_ip
  log_file="nginx.access.20150815.log"

  if [[ ! -e $local_log_dir/"$log_file" ]]; then
    mkdir -p $local_log_dir
    scp $server_ip:$remote_log_dir/$log_file $local_log_dir/$log_file
  fi
done

执行:

$./copy_log

执行后拷贝的日志如下

thin server (0.0.0.0:5200)_2015-08-15.log                                                                           100%   37MB   4.1MB/s   00:09    
thin server (0.0.0.0:5201)_2015-08-15.log                                                                           100%   37MB   4.6MB/s   00:08    
thin server (0.0.0.0:5202)_2015-08-15.log                                                                           100%   37MB   4.6MB/s   00:08    
thin server (0.0.0.0:5203)_2015-08-15.log                                                                           100%   37MB   3.7MB/s   00:10    
thin server (0.0.0.0:5204)_2015-08-15.log                                                                           100%   37MB   4.6MB/s   00:08    
thin server (0.0.0.0:5205)_2015-08-15.log                                                                           100%   37MB   4.6MB/s   00:08    
thin server (0.0.0.0:5206)_2015-08-15.log                                                                           100%   37MB   2.6MB/s   00:14    
thin server (0.0.0.0:5207)_2015-08-15.log                                                                           100%   37MB   3.7MB/s   00:10    
thin server (0.0.0.0:5200)_2015-08-15.log                                                                           100%   37MB   6.2MB/s   00:06    
thin server (0.0.0.0:5201)_2015-08-15.log                                                                           100%   37MB   4.1MB/s   00:09    
thin server (0.0.0.0:5202)_2015-08-15.log                                                                           100%   37MB   3.7MB/s   00:10    
thin server (0.0.0.0:5203)_2015-08-15.log                                                                           100%   37MB   3.1MB/s   00:12    
thin server (0.0.0.0:5204)_2015-08-15.log                                                                           100%   37MB   3.4MB/s   00:11    
thin server (0.0.0.0:5205)_2015-08-15.log                                                                           100%   37MB   3.4MB/s   00:11    
thin server (0.0.0.0:5206)_2015-08-15.log                                                                           100%   37MB   2.7MB/s   00:14    
thin server (0.0.0.0:5207)_2015-08-15.log                                                                           100%   37MB   2.7MB/s   00:14    
thin server (0.0.0.0:5200)_2015-08-15.log                                                                           100%   37MB   3.4MB/s   00:11    
thin server (0.0.0.0:5201)_2015-08-15.log                                                                           100%   37MB   4.6MB/s   00:08    
thin server (0.0.0.0:5202)_2015-08-15.log                                                                           100%   37MB   4.6MB/s   00:08    
thin server (0.0.0.0:5203)_2015-08-15.log                                                                           100%   37MB   6.2MB/s   00:06    
thin server (0.0.0.0:5204)_2015-08-15.log                                                                           100%   37MB   3.1MB/s   00:12    
thin server (0.0.0.0:5205)_2015-08-15.log                                                                           100%   37MB   3.4MB/s   00:11    
thin server (0.0.0.0:5206)_2015-08-15.log                                                                           100%   37MB   4.7MB/s   00:08    
thin server (0.0.0.0:5207)_2015-08-15.log                                                                           100%   37MB   3.7MB/s   00:10    
thin server (0.0.0.0:3201)_2015-08-15.log                                                                           100%   64KB  63.7KB/s   00:00    
thin server (0.0.0.0:3202)_2015-08-15.log                                                                           100%   63KB  63.2KB/s   00:00    
thin server (0.0.0.0:3203)_2015-08-15.log                                                                           100%   69KB  68.9KB/s   00:00    
thin server (0.0.0.0:3204)_2015-08-15.log                                                                           100%   67KB  66.9KB/s   00:00    
thin server (0.0.0.0:3205)_2015-08-15.log                                                                           100%   55KB  55.5KB/s   00:00    
thin server (0.0.0.0:3206)_2015-08-15.log                                                                           100%   63KB  63.0KB/s   00:00    
thin server (0.0.0.0:3207)_2015-08-15.log                                                                           100%   57KB  56.7KB/s   00:00    
thin server (0.0.0.0:3208)_2015-08-15.log                                                                           100%   60KB  60.4KB/s   00:00    
scp: /opt/logs/m-cms-for-tudou-tv/thin server (0.0.0.0:3201)_2015-08-15.log: No such file or directory
thin server (0.0.0.0:3202)_2015-08-15.log                                                                           100%   51KB  50.7KB/s   00:00    
thin server (0.0.0.0:3203)_2015-08-15.log                                                                           100%   51KB  50.6KB/s   00:00    
thin server (0.0.0.0:3204)_2015-08-15.log                                                                           100%   52KB  52.5KB/s   00:00    
thin server (0.0.0.0:3205)_2015-08-15.log                                                                           100%   52KB  52.0KB/s   00:00    
thin server (0.0.0.0:3206)_2015-08-15.log                                                                           100%   49KB  49.1KB/s   00:00    
thin server (0.0.0.0:3207)_2015-08-15.log                                                                           100%   52KB  52.3KB/s   00:00    
thin server (0.0.0.0:3208)_2015-08-15.log                                                                           100%   52KB  52.4KB/s   00:00    
thin server (0.0.0.0:3201)_2015-08-15.log                                                                           100%   45KB  44.9KB/s   00:00    
thin server (0.0.0.0:3202)_2015-08-15.log                                                                           100%   44KB  43.7KB/s   00:00    
thin server (0.0.0.0:3203)_2015-08-15.log                                                                           100%   46KB  45.6KB/s   00:00    
thin server (0.0.0.0:3204)_2015-08-15.log                                                                           100%   46KB  46.1KB/s   00:00    
thin server (0.0.0.0:3205)_2015-08-15.log                                                                           100%   43KB  43.4KB/s   00:00    
thin server (0.0.0.0:3206)_2015-08-15.log                                                                           100%   46KB  45.7KB/s   00:00    
thin server (0.0.0.0:3207)_2015-08-15.log                                                                           100%   44KB  44.5KB/s   00:00    
thin server (0.0.0.0:3208)_2015-08-15.log                                                                           100%   44KB  43.6KB/s   00:00    
nginx.access.20150815.log                                                                                           100%   81MB   4.3MB/s   00:19    
nginx.access.20150815.log                                                                                           100%   81MB   4.1MB/s   00:20    
nginx.access.20150815.log                                                                                           100%   81MB   2.7MB/s   00:30

 

 

(2)分析日志

分析日志的脚本如下:

#!/usr/bin/env bash

if [[ $1 == "--debug" ]]; then
  set -x
fi

LOG_PATH=~/Documents/logs/8-15
TMP_FILE=/tmp/analyse_cat.log

tv() {
  project=$1
  log_files=$LOG_PATH/$project/**/*.log

  for hour in {15..16}; do
    for min in {0..5}; do
      if [[ $hour -lt 10  ]]; then
        str="2015-08-15 0$hour:$min"
      else
        str="2015-08-15 $hour:$min"
      fi  

      request_count=$(grep "$str" $log_files | wc -l)
      response_count=$(grep "^$hour:$min" $log_files | grep Completed | wc -l)
      grep "^$hour:$min" $log_files | grep Completed > $TMP_FILE

      echo "------- $str 时总请求数: $request_count,响应数:$response_count"
      echo '响应码:'
      cut -d ' ' -f4 $TMP_FILE | sort -n | uniq -c
      # echo '响应时间:'
      # cut -d ' ' -f7 $TMP_FILE | sort -n | uniq -c | sort -nr
    done
  done
}

echo '*** Youku TV ***'
tv m-cms-for-tv

echo '*** Tubou TV ***'
tv m-cms-for-tudou-tv

执行

$ ./analyse_log

执行后的结果

~/Desktop:$ ./analyse_log 
*** Youku TV ***
------- 2015-08-15 15:0 时总请求数: 10934,响应数:10934
响应码:
  10934 200
------- 2015-08-15 15:1 时总请求数: 10908,响应数:10908
响应码:
  10908 200
------- 2015-08-15 15:2 时总请求数: 10903,响应数:10903
响应码:
  10903 200
------- 2015-08-15 15:3 时总请求数: 7184,响应数:7160
响应码:
   7160 200
------- 2015-08-15 15:4 时总请求数: 552,响应数:0
响应码:
------- 2015-08-15 15:5 时总请求数: 194063,响应数:193978
响应码:
 193978 200
------- 2015-08-15 16:0 时总请求数: 142234,响应数:142233
响应码:
 142233 200
------- 2015-08-15 16:1 时总请求数: 195244,响应数:195244
响应码:
 195244 200
------- 2015-08-15 16:2 时总请求数: 233320,响应数:233342
响应码:
 233342 200
------- 2015-08-15 16:3 时总请求数: 12787,响应数:12788
响应码:
  12788 200
------- 2015-08-15 16:4 时总请求数: 11341,响应数:11341
响应码:
  11341 200
------- 2015-08-15 16:5 时总请求数: 12470,响应数:12470
响应码:
  12470 200
*** Tubou TV ***
------- 2015-08-15 15:0 时总请求数: 5,响应数:5
响应码:
      5 200
------- 2015-08-15 15:1 时总请求数: 3,响应数:3
响应码:
      3 200
------- 2015-08-15 15:2 时总请求数: 39,响应数:39
响应码:
     39 200
------- 2015-08-15 15:3 时总请求数: 11,响应数:3
响应码:
      3 200
------- 2015-08-15 15:4 时总请求数: 31,响应数:0
响应码:
------- 2015-08-15 15:5 时总请求数: 1315,响应数:1288
响应码:
   1288 200
------- 2015-08-15 16:0 时总请求数: 9,响应数:9
响应码:
      9 200
------- 2015-08-15 16:1 时总请求数: 2,响应数:2
响应码:
      2 200
------- 2015-08-15 16:2 时总请求数: 39,响应数:39
响应码:
     39 200
------- 2015-08-15 16:3 时总请求数: 10,响应数:10
响应码:
     10 200
------- 2015-08-15 16:4 时总请求数: 2,响应数:2
响应码:
      2 200
------- 2015-08-15 16:5 时总请求数: 15,响应数:15
响应码:
     15 200

 

vim /var/log/secure

 

(1) iostat

这个命令可以检测存储设备是否工作正常,显示存储系统的细节,可以看到每个磁盘的IO情况,也可以监控到CPU 的使用状态.

14:52 [root@aaa]$ iostat
Linux 2.6.18-308.el5 (aaa) 2014年08月22日

avg-cpu: %user %nice %system %iowait %steal %idle
4.65 0.00 0.78 0.02 0.00 94.55

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.55 0.16 96.24 5548422 3391975158
sda1 0.00 0.00 0.00 1748 118
sda2 0.71 0.13 27.18 4537378 957989784
sda3 0.00 0.00 0.00 1422 304
sda4 0.00 0.00 0.00 8 0
sda5 0.83 0.03 69.06 1007434 2433984952

(2)free查看内存使用情况的综述

14:56 [root@a ]$ free -m
total used free shared buffers cached
Mem: 32168 31714 453 0 1056 26763
-/+ buffers/cache: 3894 28273
Swap: 4094 0 4094

(3)meminfo查看详细信息

14:52 [root@aaa]$ cat /proc/meminfo
MemTotal: 32940052 kB
MemFree: 464180 kB
Buffers: 1081628 kB
Cached: 27405336 kB
SwapCached: 0 kB
Active: 6155648 kB
Inactive: 24833068 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 32940052 kB
LowFree: 464180 kB
SwapTotal: 4192956 kB
SwapFree: 4192804 kB
Dirty: 16 kB
Writeback: 0 kB
AnonPages: 2501732 kB
Mapped: 32108 kB
Slab: 1428392 kB
PageTables: 26956 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 20662980 kB
Committed_AS: 4767104 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 264924 kB
VmallocChunk: 34359473015 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
Hugepagesize: 2048 kB

(4)mpstat 显示CPU状态,

可以显示所有处理器的平均状况,也可以显示统计信息

12:38 [ ]$ mpstat -P ALL
Linux 2.6.18-308.el5 (a04.api.cms.3g.b28.youku) 2014年08月23日

12时38分01秒 CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
12时38分01秒 all 4.64 0.00 0.54 0.02 0.01 0.22 0.00 94.56 67.60
12时38分01秒 0 8.58 0.00 0.93 0.05 0.05 0.71 0.00 89.68 67.60
12时38分01秒 1 3.53 0.00 0.41 0.01 0.00 0.07 0.00 95.98 0.00
12时38分01秒 2 2.84 0.00 0.33 0.01 0.00 0.05 0.00 96.76 0.00
12时38分01秒 3 3.61 0.00 0.48 0.02 0.00 0.07 0.00 95.83 0.00

12:38 [root@a04.api.cms.3g.b28.youku]$ mpstat -P 0
Linux 2.6.18-308.el5 () 2014年08月23日

12时40分43秒 CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
12时40分43秒 0 8.58 0.00 0.93 0.05 0.05 0.71 0.00 89.68 67.60

1 CPU:处理器编号,如果为all,则此行表示的是所有处理器的统计平均值
2 %usr:用户态的CPU利用率
3 %nice:具有调度优先级的用户态CPU利用率
4 %sys:内核态CPU利用率(此值不包括响应硬件中断和软件中断的时间)
5 %iowait:处理IO请求导致CPU处于IDLE状态的时间百分比
6 %irq:CPU响应硬件中断的时间比率
7 %soft:CPU响应软件中断的时间比率
8 %steal:当虚拟机监控器在服务于其他虚拟处理器时,虚拟CPU的被动等待时间比率
9 %guest:运行一个虚拟处理器所消耗的CPU时间比率

(5)netstat
-a : 显示所有socke信息
-r : 显示路由信息
-i : 显示网卡借口统计
-s : 显示网络协议统计

(6)pmap 显示进程占用的内存量,找到占用内存最多的进程.
nginx主进程内存占用情况分析

15:00 [ ]$ ps -ef | grep nginx
nobody 13609 19402 0 Aug06 ? 00:00:00 nginx: worker process
nobody 13610 19402 0 Aug06 ? 00:00:00 nginx: worker process
nobody 13611 19402 0 Aug06 ? 00:00:00 nginx: worker process
nobody 13612 19402 0 Aug06 ? 00:00:00 nginx: worker process
nobody 13613 19402 0 Aug06 ? 00:00:00 nginx: worker process
nobody 13614 19402 0 Aug06 ? 00:00:00 nginx: worker process
nobody 13615 19402 0 Aug06 ? 00:00:01 nginx: worker process
nobody 13616 19402 0 Aug06 ? 00:00:01 nginx: worker process
root 16487 16097 0 15:00 pts/1 00:00:00 grep nginx
root 19402 1 0 May05 ? 00:00:00 nginx: master process nginx

15:00 [ ]$ pmap 19402|head -n 10
19402: nginx: master process nginx
0000000000400000 1112K r-x– /opt/nginx/sbin/nginx
0000000000716000 68K rw— /opt/nginx/sbin/nginx
0000000000727000 60K rw— [ anon ]
0000000014fbc000 1080K rw— [ anon ]
0000003243200000 1204K r-x– /lib64/libcrypto.so.0.9.8e.#prelink#.6Zb7wz (deleted)
000000324332d000 2044K —– /lib64/libcrypto.so.0.9.8e.#prelink#.6Zb7wz (deleted)
000000324352c000 132K rw— /lib64/libcrypto.so.0.9.8e.#prelink#.6Zb7wz (deleted)
000000324354d000 16K rw— [ anon ]
0000003243600000 280K r-x– /lib64/libssl.so.0.9.8e