作者归档：flyinweb

iptables利用hashlimit限速

发表评论

有很长一段时间，我都在查找“端口限速”的方法，以限制流量中转的速率。

然而，搜索引擎给出的答案通常是使用iptables与tc结合，使用iptables做路由控制，使用tc做流量整型。这种方法复杂且难以维护。

同时，在GitHub上也难以查询到相关开源项目。

终于，最近找到了最高效最方便且最灵活（远不只是端口限速）的限速方式，即使用iptables的hashlimit模块。

什么是hashlimit

hashlimit是iptables的一个拓展模块，其本身是一个匹配模块，不具有其他功能，需要结合其他模块来完成复杂任务。

什么是令牌桶

hashlimit使用的算法叫做令牌桶算法。

令牌桶（token bucket），顾名思义，可以理解为有一个桶，里面装满了令牌（token）。当请求来到时，可理解为人群来到，只有拿到令牌的人才能参加活动（例如购买某种物品）。令牌桶的容量是一个固定值，且令牌桶也会在不满时以一定速率生成新的令牌，令牌的生成速率也是一个固定值。

所以，例如使用一个令牌桶来限制人群购买商品的速率，例如假设令牌桶的容量是50，生成速率是每秒100个，人群的购买欲望是无穷的（尽可能达到大的购买速率）。于是，在开放购买的瞬间，人群瞬间抢完了50个令牌（突发速率），而之后人群以每秒100人的速率购买，正好与令牌生成速度相同（限制的最高速率），同时令牌桶容量一直为0，这样一来就成功将人群购买速率限制为令牌生成速率。

继续阅读 →

DNS FLood Detector

发表评论

DNS FLood Detector 1.2

Dennis Opacki

dopacki@adotout.com

What is DNS Flood Detector? 

DNS Flood Detector was developed to detect abusive usage levels on high

traffic nameservers and to enable quick response to the use of one’s

nameserver to facilitate spam. DNS Flood Detector is distributed under the

Gnu Public License (see included LICENSE file for details).

How does it work?   

DNS Flood Detector uses libpcap (in non-promiscuous mode) to monitor

incoming  dns queries to a nameserver. The tool may be run in one of two

modes, either  daemon mode or “bindsnap” mode. In daemon mode, the tool

will alarm via syslog. In bindsnap mode, the user is able to get

near-real-time stats on usage to aid in more detailed troubleshooting.

By default, it will count dns queries directed to any address in the same

network as the primary IP address on the interface being watched; the -A,

-M, and -Q options can be used to modify this behaviour.

As of version 1.2, DNS Flood Detector can now send source IP request

data to a network-based collector as JSON. This lets you gather near

real-time information about who is using your DNS servers, and from

where. I’ve included a sample application called dns_flood_collector.pl,

which you can use to receive and report these data. The output of this

program can be easily fed into a graphing tool, such as Caida’s

plot-latlong:

http://www.caida.org/tools/visualization/plot-latlong/

How do I build it?

Execute ./configure.pl to select the appropriate make target. Then simply

type “make”.

Why was it written? 

I wrote DNS Flood Detector because the fifty or so public recursive

nameservers I am responsible for were being abused by both customers and

non-customers. DNS Flood Detector allows for prompt action when anomalous

conditions are detected.

What do I need to use it? 

You need libpcap and a little bit of patience.

What platforms does it work on?

Linux, BSDI, FreeBSD, Mac OSX, Solaris

Will it run under Windows {95,98,NT,2000,XP,2003,2008 or Win7}? 

Maybe. I haven’t tried. If it doesn’t, feel free to submit a fix.

What does it look like? 

Usage: ./dns_flood_detector [OPTION]

-i IFNAME              specify interface to listen on

-t N                   alarm at >N queries per second

-a N                   reset alarm after N seconds

-w N                   calculate stats every N seconds

-x N                   create N buckets

-m N                   mark total query rate every N seconds

-A addr                filter for specific address

-M mask                netmask for filter (in conjunction with -A)

-Q                     don’t filter by local interface address

-b                     run in foreground in bindsnap mode

-d                     run in background in daemon mode

-D                     dump dns packets (implies -b)

-v                     verbose output – use again for more verbosity

-s                     send source IP stats to collector as JSON

-z N.N.N.N             address to send stats to (default 226.1.1.2)

-p N                   UDP port to send stats to (default 2000)

-h                     display this usage information

Sample Output:

dopacki:~$ sudo ./dns_flood_detector -v -v -b -t10

[15:14:56] source [192.168.1.45] – 0 qps tcp : 24 qps udp [8 qps A] [16 qps PTR]

[15:14:56] source [10.0.24.2] – 0 qps tcp : 15 qps udp [15 qps A]

[15:15:06] source [192.168.1.45] – 0 qps tcp : 24 qps udp [8 qps A] [16 qps PTR]

[15:15:06] source [10.0.24.2] – 0 qps tcp : 15 qps udp [14 qps A]

[15:15:16] source [192.168.1.45] – 0 qps tcp : 23 qps udp [7 qps A] [15 qps PTR]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [113.96.236.11] – 0 tcp qps : 153 udp qps [151 qps A] [1 qps MX] [2 qps AAAA]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [117.185.251.211] – 0 tcp qps : 22 udp qps [20 qps A] [1 qps MX] [2 qps AAAA]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [117.185.251.192] – 0 tcp qps : 63 udp qps [57 qps A] [1 qps MX] [5 qps AAAA]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [113.96.17.165] – 0 tcp qps : 102 udp qps [100 qps A] [1 qps MX] [2 qps AAAA]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [58.60.10.12] – 0 tcp qps : 109 udp qps [107 qps A] [1 qps MX] [2 qps AAAA]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [120.204.17.12] – 0 tcp qps : 155 udp qps [154 qps A] [1 qps AAAA]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [120.204.17.67] – 0 tcp qps : 82 udp qps [81 qps A] [1 qps MX] [2 qps AAAA]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [223.167.253.12] – 0 tcp qps : 59 udp qps [57 qps A] [1 qps MX] [2 qps AAAA]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [47.252.82.250] – 1 tcp qps : 76 udp qps [76 qps A] [1 qps AAAA]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [120.204.20.12] – 0 tcp qps : 40 udp qps [40 qps A] [1 qps MX]

Feb 15 16:59:05 ns1-bd dns_flood_detector[21585]: source [112.65.192.108] – 0 tcp qps : 38 udp qps [37 qps A] [1 qps MX] [1 qps AAAA]

结合fail2ban

failregex = .* dns_flood_detector\[\d+\]: source \[<HOST>\].* \d{3,}.* (tcp|udp) qps.*

可以拦截特定IP的大量请求

Functional Programming With Python

发表评论

This article was created for those who want to use functional programming in their career or who are learning new programming paradigms.

We will examine the basics of functional programming and how to apply it in Python. Good readings!

Introduction to Functional Programming

Functional Programming is a popular programming paradigm closely linked to the mathematical foundations of computer science.

Python is not a functional programming language, but it supports it because it is a multi-paradigm programming language.

Since Python is a common and easy language, it is used in this article, but if you know a functional programming language, you can use it (Javascript, Haskell, Swift).

继续阅读 →

Linux操作系统使用 tuned/tuned-adm工具动态调优CPU性能模式

发表评论

Linux服务器CPU性能模式

环境:

  • Red Hat Enterprise Linux 4
  • Red Hat Enterprise Linux 5
  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 7

On RHEL4:

configure the cpu speed to run at MAX speed all the time by adding the following parameter into the file /etc/cpuspeed.conf and restart the cpuspeed service:

vim /etc/cpuspeed.conf
插入或修改：
OPTS="$OPTS -n -C -S \\"0 1\\""
/etc/init.d/cpuspeed restart

On RHEL 5:

add the above parameters to /etc/sysconfig/cpuspeed or use the performance governor by adding the following parameter to the /etc/sysconfig/cpuspeed file and restart the cpuspeed service:

vim /etc/sysconfig/cpuspeed
插入或修改：
GOVERNOR=performance
/etc/init.d/cpuspeed restart

On RHEL 6:

configure the performance cpuspeed governor as above for RHEL 5 and add the following kernel boot options and restart the server. More information available in the Low Latency Performance Tuning for Red Hat Enterprise Linux 6 article.

/etc/init.d/cpuspeed stop		# 开启性能模式
processor.max_cstate=1 intel_idle.max_cstate=0 idle=poll

On RHEL 7:

继续阅读 →

MySQL的Connection-Control介绍

发表评论

在Oracle和SQL Server数据库中，可以设置一些复杂的账号密码策略，例如在失败登录超过N次即可锁定账号，那么在MySQL中能否也有这样的功能呢？答案是MySQL也有类似这样的功能，只不过在MySQL中是在其登录失败超过阀值后，延迟响应时间，而不是锁定账号，在MySQL 5.6.35以后提供了Connection-Control插件用来控制客户端在登录操作连续失败一定次数后的响应延迟。该插件可有效的防止客户端暴力登录的风险(攻击)。该插件包含以下2个组件

·         CONNECTION_CONTROL：用来控制登录失败的次数及延迟响应时间

·         CONNECTION_CONTROL_FAILED_LOGIN_ATTEMPTS：该表将登录失败的操作记录至IS库中

下面我们介绍一下Connection-Control Plugins，下面实验的版本为MySQL 8.0.18。

继续阅读 →

Nmap Script脚本使用指南

发表评论

Nmap介绍
nmap是一个网络连接端扫描软件，用来扫描网上电脑开放的网络连接端。确定哪些服务运行在哪些连接端，并且推断计算机运行哪个操作系统。它是网络管理员必用的软件之一，以及用以评估网络系统安全。

在某些时候，我们大部分都使用Nmap进行扫描，然后再通过Metasploit进行漏洞利用，但是这个过程比较繁琐，一个个漏洞去验证的话还需要打开MSF去设置参数，最后run/exploit（不包括自动化，所以不要喷了）。

那么我们就要有必要认识一下Nmap的扩展脚本啦～

继续阅读 →

Python装饰器

发表评论

Python装饰器使用一个函数去包装另一个函数，本质是一个Python函数，它可以让其他函数在不需要做任何代码变动的前提下增加额外功能，装饰器的返回值也是一个函数对象。

装饰器的思想，就是把函数中除了正常行为之外的部分抽象出去，这样有很多好处，比如很容易进行代码复用，能遵守科里定律（即一次只做一件事）。

科里定律：一个变量应该代表一样东西，并且只能代表一样东西。它不应该在一种情况下代表这个意思，而在另一种情况下又代表不同的意思。它不能一次代表两样东西。它不能既是地板蜡，又是甜点上的打顶。它应该只有一个含义，并且自始至终只有一个含义。

继续阅读 →

Python 函数装饰器

发表评论

装饰器(Decorators)是 Python 的一个重要部分。简单地说：他们是修改其他函数的功能的函数。他们有助于让我们的代码更简短，也更Pythonic（Python范儿）。大多数初学者不知道在哪儿使用它们，所以我将要分享下，哪些区域里装饰器可以让你的代码更简洁。 首先，让我们讨论下如何写你自己的装饰器。

这可能是最难掌握的概念之一。我们会每次只讨论一个步骤，这样你能完全理解它。

一切皆对象

首先我们来理解下 Python 中的函数

def hi(name="yasoob"):
    return "hi " + name
 
print(hi())
# output: 'hi yasoob'
 
# 我们甚至可以将一个函数赋值给一个变量，比如
greet = hi
# 我们这里没有在使用小括号，因为我们并不是在调用hi函数
# 而是在将它放在greet变量里头。我们尝试运行下这个
 
print(greet())
# output: 'hi yasoob'
 
# 如果我们删掉旧的hi函数，看看会发生什么！
del hi
print(hi())
#outputs: NameError
 
print(greet())
#outputs: 'hi yasoob'
继续阅读 →

Python 装饰器详解

发表评论

1、闭包

要想了解装饰器，首先要了解一个概念，闭包。什么是闭包，一句话说就是，在函数中再嵌套一个函数，并且引用外部函数的变量，这就是一个闭包了。光说没有概念，直接上一个例子。

def outer(x):
    def inner(y):
        return x + y
    return inner

print(outer(6)(5))
-----------------------------
>>>11

如代码所示，在outer函数内，又定义了一个inner函数，并且inner函数又引用了外部函数outer的变量x，这就是一个闭包了。在输出时，outer(6)(5),第一个括号传进去的值返回inner函数，其实就是返回6 + y，所以再传第二个参数进去，就可以得到返回值，6 + 5。

2、装饰器

接下来就讲装饰器，其实装饰器就是一个闭包，装饰器是闭包的一种应用。什么是装饰器呢，简言之，python装饰器就是用于拓展原来函数功能的一种函数，这个函数的特殊之处在于它的返回值也是一个函数，使用python装饰器的好处就是在不用更改原函数的代码前提下给函数增加新的功能。使用时，再需要的函数前加上@demo即可。

def debug(func):
    def wrapper():
        print("[DEBUG]: enter {}()".format(func.__name__))
        return func()
    return wrapper

@debug
def hello():
    print("hello")

hello()
-----------------------------
>>>[DEBUG]: enter hello()
>>>hello

例子中的装饰器给函数加上一个进入函数的debug模式，不用修改原函数代码就完成了这个功能，可以说是很方便了。

3、带参数的装饰器

上面例子中的装饰器是不是功能太简单了，那么装饰器可以加一些参数吗，当然是可以的，另外装饰的函数当然也是可以传参数的。

def logging(level):
    def outwrapper(func):
        def wrapper(*args, **kwargs):
            print("[{0}]: enter {1}()".format(level, func.__name__))
            return func(*args, **kwargs)
        return wrapper
    return outwrapper

@logging(level="INFO")
def hello(a, b, c):
    print(a, b, c)

hello("hello,","good","morning")
-----------------------------
>>>[INFO]: enter hello()
>>>hello, good morning

如上，装饰器中可以传入参数，先形成一个完整的装饰器，然后再来装饰函数，当然函数如果需要传入参数也是可以的，用不定长参数符号就可以接收，例子中传入了三个参数。

4、类装饰器

装饰器也不一定只能用函数来写，也可以使用类装饰器，用法与函数装饰器并没有太大区别，实质是使用了类方法中的call魔法方法来实现类的直接调用。

class logging(object):
    def __init__(self, func):
        self.func = func

    def __call__(self, *args, **kwargs):
        print("[DEBUG]: enter {}()".format(self.func.__name__))
        return self.func(*args, **kwargs)

@logging
def hello(a, b, c):
    print(a, b, c)

hello("hello,","good","morning")
-----------------------------
>>>[DEBUG]: enter hello()
>>>hello, good morning

类装饰器也是可以带参数的，如下实现

class logging(object):
    def __init__(self, level):
        self.level = level

    def __call__(self, func):
        def wrapper(*args, **kwargs):
            print("[{0}]: enter {1}()".format(self.level, func.__name__))
            return func(*args, **kwargs)
        return wrapper

@logging(level="TEST")
def hello(a, b, c):
    print(a, b, c)

hello("hello,","good","morning")
-----------------------------
>>>[TEST]: enter hello()
>>>hello, good morning

好了，如上就是装饰器的一些概念和大致的用法啦，想更深入的了解装饰器还是需要自己在平时的练习和应用中多体会，本篇只是给出一个概念。

Manually configuring the /etc/resolv.conf file（RHEL7/8 ）

发表评论

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/manually-configuring-the-etc-resolv-conf-file_configuring-and-managing-networking

By default, NetworkManager on Red Hat Enterprise Linux (RHEL) 8 dynamically updates the /etc/resolv.conf file with the DNS settings from active NetworkManager connection profiles. This section describes different options on how to disable this feature to manually configure DNS settings in /etc/resolv.conf.

31.1. Disabling DNS processing in the NetworkManager configuration

This section describes how to disable DNS processing in the NetworkManager configuration to manually configure the /etc/resolv.conf file.

Procedure

  1. As the root user, create the /etc/NetworkManager/conf.d/90-dns-none.conf file with the following content by using a text editor: [main] dns=none
  2. Reload the NetworkManager service: # systemctl reload NetworkManagerNote After you reload the service, NetworkManager no longer updates the /etc/resolv.conf file. However, the last contents of the file are preserved.
  3. Optionally, remove the Generated by NetworkManager comment from /etc/resolv.conf to avoid confusion.

Verification steps

  1. Edit the /etc/resolv.conf file and manually update the configuration.
  2. Reload the NetworkManager service: # systemctl reload NetworkManager
  3. Display the /etc/resolv.conf file: # cat /etc/resolv.conf If you successfully disabled DNS processing, NetworkManager did not override the manually configured settings.

Additional resources

  • For further details, see the description of the dns parameter in the NetworkManager.conf(5) man page.

31.2. Replacing /etc/resolv.conf with a symbolic link to manually configure DNS settings

NetworkManager does not automatically update the DNS configuration if /etc/resolv.conf is a symbolic link. This section describes how to replace /etc/resolv.conf with a symbolic link to an alternative file with the DNS configuration.

Prerequisites

  • The rc-manager option is not set to file. To verify, use the NetworkManager --print-config command.

Procedure

  1. Create a file, such as /etc/resolv.conf.manually-configured, and add the DNS configuration for your environment to it. Use the same parameters and syntax as in the original /etc/resolv.conf.
  2. Remove the /etc/resolv.conf file: # rm /etc/resolv.conf
  3. Create a symbolic link named /etc/resolv.conf that refers to /etc/resolv.conf.manually-configured: # ln -s /etc/resolv.conf.manually-configured /etc/resolv.conf

Additional resources

  • For details about parameters you can set in /etc/resolv.conf, see the resolv.conf(5) man page.
  • For further details about why NetworkManager does not process DNS settings if /etc/resolv.conf is a symbolic link, see the description of the rc-manager parameter in the NetworkManager.conf(5) man page.