Sunday, October 9, 2022

Upgrade nginx on the fly - no downtime

          #first please check pid file by:

          cat /var/run/nginx.pid  

#copy new binary to /sbin/nginx and force overwrite if you don't use option "-f" you will see this error : 
cp: cannot create regular file ‘/sbin/nginx’: Text file busy
/bin/cp -f nginx /sbin/nginx 

#spawn a new nginx master/workers set

kill -s USR2 `cat /var/run/nginx.pid`

#check process

ps aux | grep nginx

# check pid 

tail -n +1 /var/run/nginx.pid*

#shut down the old master's worker

kill -s WINCH `cat /var/run/nginx.pid.oldbin`

#check 
ps aux | grep nginx

safely shut down the old master process

kill -s QUIT `cat /var/run/nginx.pid.oldbin`

Solved : ./configure: error: the Google perftools module requires the Google perftools library


./configure: error: the Google perftools module requires the Google perftools
library. You can either do not enable the module or install the library.

to fix above error run this cmd:
yum install gperftools-devel


Saturday, October 8, 2022

Understanding naxsilogs

NAXSI_FMT

NAXSI_FMT are outputed by naxsi in your errorlog :

2013/11/10 07:36:19 [error] 8278#0: *5932 NAXSI_FMT: ip=X.X.X.X&server=Y.Y.Y.Y&uri=/phpMyAdmin-2.8.2/scripts/setup.php&learning=0&vers=0.52&total_processed=472&total_blocked=204&block=0&cscore0=$UWA&score0=8&zone0=HEADERS&id0=42000227&var_name0=user-agent, client: X.X.X.X, server: blog.memze.ro, request: "GET /phpMyAdmin-2.8.2/scripts/setup.php HTTP/1.1", host: "X.X.X.X"

Here, client X.X.X.X request to server Y.Y.Y.Y did trigger the rule 42000227 in the var named user-agent in theHEADERS zone. id X might seem obscure, but you can see the meaning in naxsi_core.rules:

MainRule "str:<" "msg:html open tag" "mz:ARGS|URL|BODY|$HEADERS_VAR:Cookie" "s:$XSS:8" id:1302;

NAXSI_FMT is composed of different items :

  • ip : Client's ip
  • server : Requested Hostname (as seen in http header Host)
  • uri: Requested URI (without arguments, stops at ?)
  • learning: tells if naxsi was in learning mode (0/1)
  • vers : Naxsi version, only since 0.51
  • total_processed: Total number of requests processed by nginx's worker
  • total_blocked: Total number of requests blocked by (naxsi) nginx's worker
  • zoneN: Zone in which match happened (see "Zones" in the table below)
  • idN: The rule id that matched
  • var_nameN: Variable name in which match happened (optional)
  • cscoreN : named score tag
  • scoreN : associated named score value

Several groups of zone, id, var_name, cscore and score can be present in a single line.

NAXSI_EXLOG

NAXSI_EXLOG is a complement to naxsilogs. Along with exceptions, it contains actual content of the matched request. While NAXSI_FMT only contains IDs and location of exception, NAXSI_EXLOG provides actual content, allowing you to easily decide if it's a false positive or not.

Learning tools uses this at his advantage. Extensive log is enabled by adding the following line in your server {} section but out of your location.

set $naxsi_extensive_log 1;

This feature is provided by runtime-modifiers.

2013/05/30 20:47:05 [debug] 10804#0:*1 NAXSI_EXLOG: ip=127.0.0.1&server=127.0.0.1&uri=/&id=1302&zone=ARGS&var_name=a&content=a<>bcd
2013/05/30 20:47:05 [error] 10804#0:*1 NAXSI_FMT: ip=127.0.0.1&server=127.0.0.1&uri=/&learning=0&vers=0.50&total_processed=1&total_blocked=1&zone0=ARGS&id0=1302&var_name0=a, client: 127.0.0.1, server: , request: "GET /?a=a<>bcd HTTP/1.0", host: "127.0.0.1"

Naxsi Internal IDs

"User defined" rules are supposed to have IDs > 1000.

IDs inferior 1000 are reserved for naxsi internal rules, which are usually related to protocol sanity and things that cannot be expressed through regular expressions or string matches.

Think twice before whitelisting one of those IDs, as it might partially/totally disable naxsi.


Reference: https://github.com/nbs-system/naxsi/wiki/naxsilogs

Tuesday, October 4, 2022

Solved: utils/geo_lookup.cc:131:32: error: invalid conversion from ‘const MMDB_s’ to ‘MMDB_s’ [-fpermissive]

If you see this error when you compiled Modsecurity :

utils/geo_lookup.cc:131:32: error: invalid conversion from ‘const MMDB_s’ to ‘MMDB_s’ [-fpermissive]

because you are using old lib of maxminddb to solved it you should manual compile libmaxminddb from https://github.com/maxmind/libmaxminddb/releases: 

 sudo ./configure
sudo  make
 sudo make check
 sudo make install
 sudo ldconfig

Then you compile Modsecurity again, it should be ok .

Solved: libtoolize: command not found

Solved: 
yum install libtool 

Sunday, October 2, 2022

Solved: python error: ImportError: No module named elasticsearch

you need run below command : 
pip install elasticsearch 

That's all !  



Understanding naxsi rules

Rules are meant to search for patterns in parts of a request to detect attacks.

ie. DROP any request containing the string 'zz' in any GET or POST argument : MainRule id:424242 "str:zz" "mz:ARGS|BODY" "s:DROP";

Rules can be present at location level (BasicRule) or at http level (MainRule).

Rules have the following schema :

Everything must be quoted with double quotes, except the id part.

ID (id:...)

id:num is the unique numerical ID of the rule, that will be used in NAXSI_FMT or whitelists.

IDs inferior to 1000 are reserved for naxsi internal rules (protocol mismatch etc.)

Match Pattern

Match pattern can be a regular expression, a string match, or a call to a lib (libinjection) :

  • rx:foo|bar : will match foo or bar
  • str:foo|bar : will match foo|bar
  • d:libinj_xss : will match if libinjection says it's XSS (>= 0.55rc2)
  • d:libinj_sql : will match if libinjection says it's SQLi (>= 0.55rc2)

Using plain string match when possible is recommended, as it's way faster. All strings must be lowercase, since naxsi's matches are case insensitive.

Score (s:...)

s is the score section. You can create "named" counters: s:$FOOBAR:4 will increase counter $FOOBAR value by 4. One rule can increase several scores: s:$FOO:4,$BAR:8 will increase both $FOO by 4 and $BAR by 8. A rule can as well directly specifiy an action such a BLOCK (blocks the request in non-learning mode) or DROP (blocks the request even in learning mode) Named scores are later handled by CheckRules.

MatchZone (mz:...)

Please refer to Match Zones for details.

mz is the match zone, defining which part of the request will be inspected by the rule.

In rules, all matchzones but $URL*: are treated as OR conditions :

MainRule id:4242 str:z "mz:$ARGS_VAR:X|BODY";

pattern 'z' will be searched in GET var 'X' and all BODY vars.

MainRule id:4242 str:z "mz:$ARGS_VAR:X|BODY|$URL_X:^/foo";

pattern 'z' will be searched in GET var 'X' and all BODY vars as long as URL starts with /foo.

Starting from naxsi 0.55rc0, for unknown content-types, you can use the RAW_BODY match-zone. RAW_BODY rules looks like that:

MainRule id:4241 s:DROP str:RANDOMTHINGS mz:RAW_BODY;

Rules in the RAW_BODY zone will only applied when:

  • The Content-type is unknown (which means naxsi doesn't know how to properly parse the request)
  • id 11 (which is the internal blocking rule for 'unknown content-type') is whitelisted.

Then, the full body (url decoded and with null-bytes replaced by '0') is passed to this set of rules. The full body is matched again the regexes or string matches.

Whitelists for RAW_BODY rules are actually written just like normal body rules, such as:

BasicRule wl:4241 "mz:$URL:/rata|BODY";

Human readable message (msg:...)

msg is a string describing the pattern. This is mostly used for analyzing and to have some human-understandable text.

Negative Keyword (negative)

negative is a keyword that can be used to make a negative rule. Score is applied when the rule doesn't match :

MainRule negative "rx:multipart/form-data|application/x-www-form-urlencoded" "msg:Content is neither mulipart/x-www-form.." "mz:$HEADERS_VAR:Content-type" "s:$EVADE:4" id:1402;

Reference : https://github.com/nbs-system/naxsi/wiki/rules-bnf

Solved : nginx: [emerg] Naxsi-Config : Incorrect line MainRule rx:select in /etc/nginx/waf/naxsi_core.rules:23

If you see this error on Redhat( or Centos) when you are testing config with nginx ( nginx -t or reload config) : 

nginx: [emerg] Naxsi-Config : Incorrect line MainRule rx:select|union|update|delete|insert|table|from|ascii|hex|unhex|drop|load_file|substr|group_concat|dumpfile (./naxsi/naxsi_src/naxsi_skeleton.c/973)... in /etc/nginx/waf/naxsi_core.rules:23

This is solution to solved it : 

1. Get naxsi from this url: https://github.com/wargio/naxsi by cmd: 
git clone --recurse-submodules https://github.com/wargio/naxsi.git

2. Then re-compile modules follow the steps below: 

- first cd into nginx directory then : 

./configure  --with-compat --add-dynamic-module=./naxsi/naxsi_src 

 - and run : 

make modules      

 - After that copy module file into nginx modules directory: cp objs/ngx_http_naxsi_module.so /usr/lib64/nginx/modules/

Thanks Wargio ! 

Done ! 

Saturday, October 1, 2022

Create nginx systemd file

create file : /lib/systemd/system/nginx.service with below content: 

[Unit]
Description=The NGINX HTTP and reverse proxy server
After=syslog.target network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t
ExecStart=/usr/sbin/nginx
ExecReload=/usr/sbin/nginx -s reload
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

then run :

systemctl daemon-reload && systemctl enable nginx && systemctl start nginx 

Done ! 

Resolved: ./naxsi/naxsi_src/naxsi_runtime.c:188:8: error: unknown type name ‘pcre2_match_data’

When you use naxsi from : https://github.com/nbs-system/naxsi and compile it on ubuntu you would see this error: 

./naxsi/naxsi_src/naxsi_runtime.c:188:8: error: unknown type name ‘pcre2_match_data

  188 | static pcre2_match_data       *ngx_pcre2_match_data;

      |        ^~~~~~~~~~~~~~~~

./naxsi/naxsi_src/naxsi_runtime.c: In function ‘ngx_pcre2_exec’:

./naxsi/naxsi_src/naxsi_runtime.c:216:13: error: implicit declaration of function ‘pcre2_match_data_free’ [-Werror=implicit-function-declaration]

  216 |             pcre2_match_data_free(ngx_pcre2_match_data);

      |             ^~~~~~~~~~~~~~~~~~~~~

./naxsi/naxsi_src/naxsi_runtime.c:220:32: error: implicit declaration of function ‘pcre2_match_data_create’ [-Werror=implicit-function-declaration]

  220 |         ngx_pcre2_match_data = pcre2_match_data_create(size / 3, NULL);

      |                                ^~~~~~~~~~~~~~~~~~~~~~~

./naxsi/naxsi_src/naxsi_runtime.c:220:30: error: assignment to ‘int *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]

  220 |         ngx_pcre2_match_data = pcre2_match_data_create(size / 3, NULL);

      |                              ^

./naxsi/naxsi_src/naxsi_runtime.c:223:18: error: PCRE2_ERROR_NOMEMORY’ undeclared (first use in this function); did you mean ‘PCRE_ERROR_NOMEMORY’?

  223 |             rc = PCRE2_ERROR_NOMEMORY;

      |                  ^~~~~~~~~~~~~~~~~~~~

      |                  PCRE_ERROR_NOMEMORY

./naxsi/naxsi_src/naxsi_runtime.c:223:18: note: each undeclared identifier is reported only once for each function it appears in

./naxsi/naxsi_src/naxsi_runtime.c:228:10: error: implicit declaration of function ‘pcre2_match’ [-Werror=implicit-function-declaration]

  228 |     rc = pcre2_match(re, str, len, tmp_idx, 0, ngx_pcre2_match_data, NULL);

      |          ^~~~~~~~~~~

./naxsi/naxsi_src/naxsi_runtime.c:234:9: error: implicit declaration of function ‘pcre2_get_ovector_count’ [-Werror=implicit-function-declaration]

  234 |     n = pcre2_get_ovector_count(ngx_pcre2_match_data);

      |         ^~~~~~~~~~~~~~~~~~~~~~~

./naxsi/naxsi_src/naxsi_runtime.c:235:10: error: implicit declaration of function ‘pcre2_get_ovector_pointer’ [-Werror=implicit-function-declaration]

  235 |     ov = pcre2_get_ovector_pointer(ngx_pcre2_match_data);

      |          ^~~~~~~~~~~~~~~~~~~~~~~~~

./naxsi/naxsi_src/naxsi_runtime.c:235:8: error: assignment to ‘size_t *’ {aka ‘long unsigned int *’} from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]

  235 |     ov = pcre2_get_ovector_pointer(ngx_pcre2_match_data);

      |        ^



To resolved this error , you should use naxsi from : https://github.com/wargio/naxsi  by this cmd below : 

git clone --recurse-submodules https://github.com/wargio/naxsi.git

Then compile it again , It should be ok ! Thanks wargio !