分类 Ubuntu 下的文章

“Ubuntu是以桌面应用为主的Linux发行版,基于Debian。Ubuntu有三个正式版本,包括桌面版、服务器版及用于物联网设备和机器人的Core版。从17.10版本开始,Ubuntu以GNOME为默认桌面环境。 Ubuntu是著名的Linux发行版之一,也是目前最多用户的Linux版本。 ”

列出当前存在的网络接口

tshark -D
  1. ciscodump (Cisco remote capture)
  2. dpauxmon (DisplayPort AUX channel monitor capture)
  3. randpkt (Random packet generator)
  4. sdjournal (systemd Journal Export)
  5. sshdump (SSH remote capture)
  6. udpdump (UDP Listener remote capture)

    sudo tshark -D

Running as user "root" and group "root". This could be dangerous.

  1. eth0
  2. any
  3. lo (Loopback)
  4. docker0
  5. nflog
  6. nfqueue
  7. usbmon1
  8. ciscodump (Cisco remote capture)
  9. dpauxmon (DisplayPort AUX channel monitor capture)
  10. randpkt (Random packet generator)
  11. sdjournal (systemd Journal Export)
  12. sshdump (SSH remote capture)
  13. udpdump (UDP Listener remote capture)

tshark对指定网卡监听,抓包

sudo tshark -i <interface>

抓取网卡eth0的流量并写入capture123.pcap

tshark -i eth0 -w capture123.pcap

读取之前的文件capture123.pcap

tshark -i eth0 -r capture123.pcap

抓取网卡eth0的流量10分钟

tshark -i eth0 -a duration:600

注: 默认时间单位为秒

抓取网卡eth0的10000个数据包

tshark -c 10000 -i eth0

抓取网卡eth0涉及192.168.1.1的流量报文

tshark -i eth0 -f "host 192.168.1.1"

注: 与wireshark、tcpdump一致,均使用BPF过滤表达式
抓取网卡eth0指定协议的流量报文

tshark -i eth0 -f "<协议名>"

协议名可以为: tcp, udp, dns, icmp, http等

实时打印当前mysql查询语句

tshark -s 512 -i eth1 -n -f 'tcp dst port 3306' -R 'mysql.query' -T fields -e mysql.query

说明:

-s 512 :只抓取前512个字节数据
-i eth0 :监听eth0网卡
-n :禁止域名解析
-f ‘tcp dst port 3306’ :只捕捉协议为tcp,目的端口为3306的数据包
-R ‘mysql.query’ :过滤出mysql.query查询语句的报文
-T fields -e mysql.query :打印mysql查询语句
实时打印当前http请求的url(包括域名)

tshark -s 512 -i eth1 -n -f 'tcp dst port 8000' -R 'http.host and http.request.uri' -T fields -e http.host -e http.request.uri -l | tr -d 't'

说明:

-s 512 :只抓取前512个字节数据
-i eth1 :监听eth1网卡
-n :禁止网络对象名称解析
-f ‘tcp dst port 8000’ :只捕捉协议为tcp,目的端口为8000的数据包
-R ‘http.host and http.request.uri’ :过滤出http.host和http.request.uri
-T fields -e http.host -e http.request.uri :打印http.host和http.request.uri
-l :输出到标准输出
读取之前抓包文件进行报文数据分析
需要从抓包的文件evidence04.pcap中提取出报文相关数据信息,如时间、源IP、目的IP、协议名、源Port、標Port、包大小等信息,最后输出到csv文件。

tshark -r evidence.pcap -T fields -e frame.time_relative -e ip.src -e ip.dst -e ip.proto -e tcp.srcport -e tcp.dstport -e frame.len -E header=n -E separator=, -E quote=n -E occurrence=f > output.csv

说明:

-r evidence.pcap 需要分析的报文记录文件(pcap格式)
-T fields 输出格式,选fields按字段,也可以选json等其他格式,需结合-e 及 -E使用
-e frame.time_relative 取出封包的相对时间
-e ip.src 提取源IP
-e ip.dst 提取目的IP
-e ip.proto 提取协议名
-e tcp.srcport 提取源Port
-e tcp.dstport 提取目的Port
-e frame.len 提取数据帧大小
-E header=n 是否输出字段名称(cvs的第1行)
-E separator=, 指定分割符,/t是tab,/s是一格空格
-E quote=n 指定是否对字段用引号,d是双引号,s是单引号,n是不用
-E occurrence=f 多值时是否保留,f是第一个值,l是最后一个值,a是所有值都列出,默认全部
output.csv 输出文件路径及名称
DNS报文过滤
使用tshark过滤dns cap包中源ip、目的ip、request请求

tshark -r test.cap -T fields -e frame.time -e ip.src -e ip.dst -e dns.qry.name -R 'udp.dstport==53 || dns'
说明:

-r test.pcap 需要分析的报文记录文件(pcap格式)
-T fields 输出格式,选fields按字段,也可以选json等其他格式,需结合-e 及 -E使用
-e frame.time 提取数据帧时间
-e ip.src 提取源IP
-e ip.dst 提取目的IP
-e dns.qry.name 提取dns查询的域名信息
-R 'udp.dstport==53 || dns' 显示过滤,仅对udp目标端口为53或者dns协议的报文进行处理
默认直接显示在终端上,不记录文件。

tshark: Couldn't run /usr/bin/dumpcap in child process: Permission denied
解决办法:
On Debian and Debian derivatives such as Ubuntu, if you have installed Wireshark from a package, 1.try running

sudo dpkg-reconfigure wireshark-common

selecting "<Yes>" in response to the question

2.adding yourself to the "wireshark" group by running

sudo usermod -a -G wireshark {your username}
  1. then logging out and logging back in again.

tshark 示例

tshark -T ek -e data.data -i eth0 "udp port 12345"

Capturing on 'eth0'
{"index" : {"_index": "packets-2022-03-03", "_type": "pcap_file"}}
{"timestamp" : "1646291304083", "layers" : {"data_data": ["ff:ff:00:b0:e1:3e:c9:42:6f:32:01:19:21:23:07:10:27:12:49:bd:ad:cb:d5:32:01:00:01:00:40:32:01:62:19:61:61:29:82:20:19:10:21:20:29:10:21:cb:d5:42:31:53:30:36:30:00:00:00:00:00:00:01:00:00:2e:11:0f:04:02:00:1c:00:00:05:d0:a1:d0:cd:c6:fb:b3:b5:00:00:00:00:00:00:00:00:32:34:31:35:31:34:54:00:00:00:00:00:00:00:00:00:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:bd:ad:cb:d5:32:01:00:01:17:40:32:01:19:21:23:07:10:27:12:49:20:19:10:21:20:29:10:21:cb:d5:42:31:53:30:36:30:00:00:00:00:00:00:01:ff:ff:ff:ff:ff:ff:ff:20:01:00:00:00:00:85:59"]}}
{"index" : {"_index": "packets-2022-03-03", "_type": "pcap_file"}}
{"timestamp" : "1646291304084", "layers" : {"data_data": ["ff:ff:00:10:f1:00:3e:c9:42:6f:32:01:19:21:23:07:10:27:12:49:61:b9"]}}

tshark 高端用法
tshark结合elasticsearch分析WIFI网络流量
https://harrunisk.github.io/2018-09-12-ElasticWifiAnalysisEn/

  • Tshark captures wireless packets by using filters.

    tshark -a duration:600 -i phy0.mon -t ad -t ad -lT fields -E separator=, -E quote=d -e _ws.col.Time -e wlan.fc.type -e wlan.fc.type_subtype -e radiotap.dbm_antsignal -e frame.len -e radiotap.datarate > tshark.csv

  • Tshark writes captured wireless packets as .csv.
  • Filebeat listens .csv file sends to Logstash.
  • Logstash filters data again and sends to Elasticsearch.
  • Analyze data by using Kibana. Dashboard, graph etc.

架构图

tshark 输出csv格式数据
命令

tshark -a duration:6 -t ad -t ad  -Tfields -E separator=, -E quote=d -e ip.src -e ip.dst -e data.len -i eth0 "udp port 12345"

结果

"61.54.103.229","172.18.73.109","182"
"172.18.73.109","61.54.103.229","22"
"61.153.149.53","172.18.73.109","182"
"172.18.73.109","61.153.149.53","22"
"223.112.138.246","172.18.73.109","182"
"172.18.73.109","223.112.138.246","22"

tshark支持将pcap报文分析后生成json文件导入elasticsearch,同时支持elasticsearch的批量导入接口_bulk的格式,命令如下:

tshark -r test_trace.pcap -T ek > test_trace.pcap.json

之后可以将json文件通过curl导入。

curl -s -H "Content-Type: application/x-ndjson" -XPOST "localhost:9200/foo/_bulk" --data-binary "@/Users/test-elastic/test_trace.pcap.json"

网络流量回溯分析(https://paper.seebug.org/427/

ubuntu20 安装最新版的wireshark/tshark
默认的tshark 版本还是2.6.10

tshark -v

TShark (Wireshark) 2.6.10 (Git v2.6.10 packaged as 2.6.10-1~ubuntu18.04.0)

tshark -v 

TShark (Wireshark) 3.2.3 (Git v3.2.3 packaged as 3.2.3-1)

安装最新的
可选

sudo apt-get install software-properties-common
sudo add-apt-repository ppa:wireshark-dev/stable
sudo apt update
sudo apt -y install wireshark tshark
tshark -v

TShark (Wireshark) 3.4.8 (Git v3.4.8 packaged as 3.4.8-1~ubuntu20.04.0+wiresharkdevstable1)

wireshark/tshark相关权限配置

sudo usermod -a -G wireshark $USER
sudo chgrp wireshark /usr/bin/dumpcap
sudo chmod 750 /usr/bin/dumpcap
sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/dumpcap

Then verify it by submitting command line :

sudo getcap /usr/bin/dumpcap

/usr/bin/dumpcap = cap_net_admin,cap_net_raw+eip
tshark -G elastic-mapping
Starting with Wireshark 3.0.0rc1, TShark can now generate an Elasticsearch mapping file by using the -G elastic-mapping option.
Analyzing Network Packets with Wireshark, Elasticsearch, and Kibana
https://www.elastic.co/cn/blog/analyzing-network-packets-with-wireshark-elasticsearch-and-kibana

Rsyslog简介

Rsyslog 是一个快速处理收集系统日志的程序,提供了高性能、安全功能和模块化设计。rsyslog 是syslog 的升级版,它将多种来源输入输出转换结果到目的地。reliable syslog over TCP, SSL/TLS and RELP

Rsyslog can be configured in a client/server model. When configured as a client, it sends logs to a remote server over the network via TCP/UDP protocols. As a server, it receives logs over the network from remote client on port 514 TCP/UDP or any custom port on which it is configured to listen on.

Install Rsyslog on Ubuntu 20.04
Rsyslog is the default syslogd on Debian systems and is usually installed on Ubuntu 20.04 by default.
You can verify this by checking the version of installed rsyslog.

apt list -a rsyslog

Listing... Done
rsyslog/bionic,bionic-updates,now 8.32.0-1ubuntu4 amd64 [installed]

rsyslogd -v

rsyslogd 8.32.0, compiled with:

Setup Rsyslog Server on Ubuntu 20.04
配置Rsyslog 服务器模式
Open the ryslog configuration file for editing;

vim /etc/rsyslog.conf

找到#### MODULES ####部分,修改成如下所示:

# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="5140")

# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="5140")

重新启动rsyslog

sudo systemctl restart rsyslog

查看监听是否正常

netstat -na |grep 0:514

tcp 0 0 0.0.0.0:5140 0.0.0.0:* LISTEN
udp 0 0 0.0.0.0:5140 0.0.0.0:*

ss -4altunp | grep 514

udp UNCONN 0 0 0.0.0.0:5140 0.0.0.0:*
tcp LISTEN 0 25 0.0.0.0:5140 0.0.0.0:*

如果有防火墙,打开防火墙

ufw allow 5140/udp
ufw allow 5140/tcp

限制允许访问的IP地址/域名

vim /etc/rsyslog.conf

...

###########################
#### GLOBAL DIRECTIVES ####
###########################
# $AllowedSender - specifies which remote systems are allowed to send syslog messages to rsyslogd
$AllowedSender UDP, 192.168.57.0/24, [::1]/128, *.example.net, servera.example.com
$AllowedSender TCP, 192.168.58.0/24, [::1]/128, *.example.net, servera.example.com

配置保存的文件/模板

vim /etc/rsyslog.conf
# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="5140")

#Custom template to generate the log filename dynamically based on the client's IP address.
$template RemInputLogs, "/var/log/remotelogs/%FROMHOST-IP%/%PROGRAMNAME%.log"
*.* ?RemInputLogs

检查配置文件是否正确

rsyslogd -f /etc/rsyslog.conf -N1

rsyslogd: version 8.2001.0, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.

没有问题,重新Rsyslog服务

systemctl restart rsyslog

Rsyslogd is now ready to receive logs from remote hosts.

配置Rsyslog客户端转发日志到服务器
验证tcp/udp远程连接
Verify Remote Rsyslog Server Ports Connection
To verify connectivity to remote rsyslog server TCP port 5140, run the command below;

telnet 192.168.57.3 5140

Verify connectivity to UDP port 5140. Since you cannot telnet to UDP port 514, use netcat command. On the server, run the command below;

nc -ul 5140

On the client, run the command below, press ENTER and type anything. You should be able to see what you type on the server.

nc -u 192.168.57.3 5140

配置客户端

vim /etc/rsyslog.conf

To send authentication logs over port 5140/UDP, add the following line at the end of the file.

# Send logs to remote syslog server over UDP
auth,authpriv.* @192.168.57.3:5140

To send all logs over port 5140/TCP, add the following line at the end of the file.

# Send logs to remote syslog server over TCP 5140
*.* @@192.168.57.3:5140

如果你的日志非常重要,不接受丢失,就配置下面参数,保存到磁盘上面。

# Define Disk Queue Buffer in case the server goes down
$ActionQueueFileName queue # define a file name for disk assistance.
$ActionQueueMaxDiskSpace 1g  # The maximum size that all queue files together will use on disk.
$ActionQueueSaveOnShutdown on  # specifies that data should be saved at shutdown
$ActionQueueType LinkedList  # holds enqueued messages in memory which makes the process very fast. 
$ActionResumeRetryCount -1  # prevents rsyslog from dropping messages when retrying to connect if server is not responding,

Restart the rsyslog service on the client.

systemctl restart rsyslog

You can now log out of the client and login again. The authentication logs should be available on rsyslog server.

Login to the Rsyslog server and verify the same.

ls /var/log/remotelogs/

send specific logs to remote server

The following sample monitors two files. If you need just one, remove the second one. If you need more, add them according to the sample ;). This code must be placed in /etc/rsyslog.conf (or wherever your distro puts rsyslog’s config files). Note that only commands actually needed need to be specified. The second file uses less commands and uses defaults instead.

module(load="imfile" PollingInterval="10") #needs to be done just once

# File 1
input(type="imfile"
      File="/path/to/file1"
      Tag="tag1"
      Severity="error"
      Facility="local7")

# File 2
input(type="imfile"
      File="/path/to/file2"
      Tag="tag2")

# ... and so on ... #

relp rsyslog 相关
relp服务器端配置
https://www.rsyslog.com/doc/v8-stable/configuration/modules/imrelp.html
前提条件

apt-get install rsyslog-relp

Edit /etc/rsyslog.conf to enable RELP input module. For example, to listen on port 2514:

module(load="imrelp") # needs to be done just once
input(type="imrelp" port="2514" maxDataSize="10k")

配置tls支持
Receive RELP traffic via TLS¶
This receives RELP traffic via TLS using the recommended “openssl” library. Except for encryption support the scenario is the same as in Example 1.

Certificate files must exist at configured locations. Note that authmode “certvalid” is not very strong - you may want to use a different one for actual deployments. For details, see parameter descriptions.

module(load="imrelp" tls.tlslib="openssl")
input(type="imrelp" port="2514" maxDataSize="10k"
             tls="on"
             tls.cacert="/tls-certs/ca.pem"
             tls.mycert="/tls-certs/cert.pem"
             tls.myprivkey="/tls-certs/key.pem"
             tls.authmode="certvalid"
             tls.permittedpeer="rsyslog")

客户端配置
Sending msgs with omrelp
The following sample sends all messages to the central server “centralserv” at port 2514 (note that that server must run imrelp on port 2514).

module(load="omrelp")
action(type="omrelp" target="centralserv" port="2514")

Sending msgs with omrelp via TLS
This is the same as the previous example but uses TLS (via OpenSSL) for operations.

Certificate files must exist at configured locations. Note that authmode “certvalid” is not very strong - you may want to use a different one for actual deployments. For details, see parameter descriptions.

module(load="omrelp" tls.tlslib="openssl")
action(type="omrelp"
             target="centralserv" port="2514" tls="on"
             tls.cacert="tls-certs/ca.pem"
             tls.mycert="tls-certs/cert.pem"
             tls.myprivkey="tls-certs/key.pem"
             tls.authmode="certvalid"
             tls.permittedpeer="rsyslog")

旧命令用下面这个
This module uses old-style action configuration to keep consistent with the forwarding rule. So far, no additional configuration directives can be specified. To send a message via RELP, use

*.*  :omrelp:<server>:<port>;<template>

rsyslog 新命令都是使用action的方式
Use the following configuration in /etc/rsyslog.conf or create a file with the following content in the /etc/rsyslog.d/ directory:

*.* action(type="omfwd"
      queue.type="linkedlist"
      queue.filename="example_fwd"
      action.resumeRetryCount="-1"
      queue.saveOnShutdown="on"
      target="example.com" port="6514" protocol="tcp"
     )

Sample syslog.conf 配置tcp 使用 tls通信
Keep in mind that this rsyslog.conf accepts messages via TCP, only. The only other source accepted is messages from the server itself.

module(load="imuxsock") # local messages
module(load="imtcp" # TCP listener
    StreamDriver.Name="gtls"
    StreamDriver.Mode="1" # run driver in TLS-only mode
    StreamDriver.Authmode="anon"
    )

# make gtls driver the default and set certificate files
global(
    DefaultNetstreamDriver="gtls"
    DefaultNetstreamDriverCAFile="/path/to/contrib/gnutls/ca.pem"
    DefaultNetstreamDriverCertFile="/path/to/contrib/gnutls/cert.pem"
    DefaultNetstreamDriverKeyFile="/path/to/contrib/gnutls/key.pem"
    )

    # start up listener at port 6514
    input(
    type="imtcp"
    port="6514"
    )

Be sure to safeguard at least the private key (machine-key.pem)! If some third party obtains it, you security is broken!

omfwd用法

Example 1
The following command sends all syslog messages to a remote server via TCP port 10514.

action(type="omfwd" Target="192.168.2.11" Port="10514" Protocol="tcp" Device="eth0")

Example 2
In case the system in use has multiple (maybe virtual) network interfaces network namespaces come in handy, each with its own routing table. To be able to distribute syslogs to remote servers in different namespaces specify them as separate actions.

action(type="omfwd" Target="192.168.1.13" Port="10514" Protocol="tcp" NetworkNamespace="ns_eth0.0")
action(type="omfwd" Target="192.168.2.24" Port="10514" Protocol="tcp" NetworkNamespace="ns_eth0.1")
action(type="omfwd" Target="192.168.3.38" Port="10514" Protocol="tcp" NetworkNamespace="ns_eth0.2")

rsyslog central logging to hostname.log excluding localhost

# Add on top of the RULES statement

$template remote, "/var/log/%HOSTNAME%.log"
if ($fromhost-ip != "127.0.0.1" ) then -?remote
& stop

storing-messages-from-a-remote-system-into-a-specific-file
Config Statements

$ModLoad imtcp
$InputTCPServerRun 10514
# do this in FRONT of the local/regular rules
if $fromhost-ip startswith '192.0.1.' then /var/log/network1.log
& ~
if $fromhost-ip startswith '192.0.2.' then /var/log/network2.log
& ~
# local/regular rules, like
*.* /var/log/syslog.log

How it works
It is important that the rules processing the remote messages come before any rules to process local messages. The if’s above check if a message originates on the network in question and, if so, writes them to the appropriate log. The next line (“& ~”) is important: it tells rsyslog to stop processing the message after it was written to the log. As such, these messages will not reach the local part. Without that “& ~”, messages would also be written to the local files.

Also note that in the filter there is a dot after the last number in the IP address. This is important to get reliable filters. For example, both of the addresses “192.0.1.1” and “192.0.10.1” start with “192.0.1” but only one actually starts with “192.0.1.”!

参考文档:https://blog.csdn.net/McwoLF/article/details/110121026
https://www.rsyslog.com/doc/v8-stable/configuration/modules/omrelp.html
https://gist.github.com/drmalex07/bb178d61f800488446d22de4301160f1
https://www.rsyslog.com/doc/v8-stable/tutorials/tls_cert_server.html
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/sec-using_the_new_syntax_for_rsyslog_queues
https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html

在终端下输入如下命令

ssh -N -v username@hostip -D 127.0.0.1:7070

把其中的 username , hostip 替换成你自已的内容。

第一次运行此命令需要输入 yes 来接受证书,最后输入 SSH 密码。如果你不想每次都输入密码的话,可以采用证书认证方式。

cat /etc/os-release 

PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

openssl version

OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.1 14 Dec 2021)

arm-linux-gnueabihf-gcc 4.9.4

wget https://releases.linaro.org/components/toolchain/binaries/4.9-2017.01/arm-linux-gnueabihf/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf.tar.xz

arm-linux-gnueabihf-gcc 5.4.1

wget https://releases.linaro.org/components/toolchain/binaries/5.4-2017.01/arm-linux-gnueabihf/gcc-linaro-5.4.1-2017.01-x86_64_arm-linux-gnueabihf.tar.xz

arm-linux-gnueabihf-gcc 6.3.1

wget https://releases.linaro.org/components/toolchain/binaries/6.3-2017.02/arm-linux-gnueabihf/gcc-linaro-6.3.1-2017.02-x86_64_arm-linux-gnueabihf.tar.xz

aarch64-linux-gnu-gcc 5.4.1

wget https://releases.linaro.org/components/toolchain/binaries/5.4-2017.01/aarch64-linux-gnu/gcc-linaro-5.4.1-2017.01-x86_64_aarch64-linux-gnu.tar.xz

aarch64-linux-gnu-gcc 7.5.0

wget https://releases.linaro.org/components/toolchain/binaries/latest-7/aarch64-linux-gnu/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tar.xz