simonchiang/goaccess
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v1.10.2 patches: continent persistence, city markers, dynamic map center

Browse Source 
C changes:
- gstorage.c: encode continent into root key for --persist --restore survival
- gholder.c: decode continent from root + lat/lng storage
- commons.h: add lat/lng fields to GMetrics struct
- geoip2.c/geoip1.h: extract coordinates from MMDB
- json.c: output lat/lng in geolocation JSON

JS changes:
- Remove cityIndex guard for static HTML city dots
- TopoJSON lat/lng fallback for city coordinates
- updateMap fallback city collection
- Dynamic orthographic center based on visitor count
- Mercator stays at [0,15] for full world view

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
simonchiang
2026-06-25 11:38:09 +08:00
commit 7436bf6e2e
133 changed files with 68771 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AUTHORS
+174
View File
@@ -0,0 +1,174 @@
GoAccess was designed and developed by Gerardo Orellana <goaccess@prosoftcorp.com>
Special thanks to the following individuals for their great contributions:
* 0bi-w6n-K3nobi <71027865+0bi-w6n-K3nob@users.noreply.github.com>
* Aaditya Bagga <noreply@github.com>
* abgit <francisco.j.azevedo@gmail.com>
* Abhishek Deshpande <abhishek@Abhisheks-MacBook-Air-2.local>
* Adam Monsen <haircut@gmail.com>
* Adam Števko <adam.stevko@gmail.com>
* Adam Weinberger <adamw@adamw.org>
* Adrian <totcha@gazeta.pl>
* Ainsley Ellis <ainsleymae@proton.me>
* aishikoyo <aixxx033@umn.edu>
* Alan Placidina <noreply@github.com>
* Alexander Eifler <noreply@github.com>
* Alexandre GUIOT--VALENTIN <contact@alexandregv.fr>
* Alexandre Perrin <noreply@github.com>
* Alexis Lefebvre <alexislefebvre+github@gmail.com>
* Ali Homafar <home.isfar@gmail.com>
* allofmex <allofmex@web.de>
* Amos Hayes <ahayes@gcrc.carleton.ca>
* A. Nackov <anackov@gmail.com>
* Anders Johansson <47452862+tellustheguru@users.noreply.github.com>
* Andreas Sommer <andreas.sommer@ppro.com>
* Andreas Weigel <andreaswe@securepoint.de>
* Andrew Kvalheim <Andrew@Kvalhe.im>
* Andrew Minion <andrew@andrewrminion.com>
* Antonio Terceiro <terceiro@softwarelivre.org>
* Arnaud Rebillout <elboulangero@gmail.com>
* Arnie97 <arnie97@gmail.com>
* as0n <n.was.here@gmail.com>
* Aslak Raanes <noreply@github.com>
* Axel Wehner <mail@axelwehner.de>
* azio7 <azio7@outlook.com>
* bbbboom <bwdyxg@gmail.com>
* Bbertatum <bertrand@2backup.fr>
* Benjamin Bach <benjamin@overtag.dk>
* Bjørnar Hansen <noreply@github.com>
* Bob Black <git@riblack.com>
* Bo Cai <charpty@gmail.com>
* Brandon Coleman <metrix1978@gmail.com>
* Branimir Ri\v{c}ko <rickobranimir@gmail.com>
* Carlos Duelo <noreply@github.com>
* Celso Providelo <celso.providelo@gmail.com>
* ChangMo Yang <dcyang@unist.ac.kr>
* Chang Zhao <noreply@github.com>
* Chilledheart <rwindz0@gmail.com>
* Chris Downs <noreply@github.com>
* Christian Göttsche <cgzones@googlemail.com>
* Christian Hermann <c.hermann@bitbeans.de>
* Christian Moelders <christian.moelders@storeplus.de>
* Christopher Meng <i@cicku.me>
* Clément Hermann <nodens@nodens.org>
* cristianpb <felipebrokate@gmail.com>
* Cthulhux <github@tuxproject.de>
* Daniel Aleksandersen <code@daniel.priv.no>
* Daniel Aleksandersen <noreply@github.com>
* Daniel (dmilith) Dettlaff <dmilith@me.com>
* Danila Vershinin <noreply@github.com>
* Danny Kirkham <danny@kirkham.dev>
* Darek Kay <hello@darekkay.com>
* David Carlier <devnexen@gmail.com>
* David Geistert <d3f3kt@scripts4u.net>
* d_dandrew <dragonandy@foxmail.com>
* ElXreno <elxreno@gmail.com>
* Enrique Becerra <kabeza@gmail.com>
* evitalis <evitalis@users.noreply.github.com>
* Felix Häberle <34959078+felixhaeberle@users.noreply.github.com>
* Florian Forster <ff@octo.it>
* forDream <noreply@github.com>
* fqbuild <fqbuild@FutureQuest.net>
* Frederic Cambus <fcambus@users.sourceforge.net>
* Func <Funcer@outlook.com>
* gemmaro <gemmaro.dev@gmail.com>
* Genki Sugawara <sugawara@cookpad.com>
* Gerald Combs <gerald@zing.org>
* Geraldo Alves <junior.alves.2007@gmail.com>
* gitqlt <noreply@github.com>
* Hiroki Kamino <46459949+err931@users.noreply.github.com>
* holys <chendahui007@gmail.com>
* Izzy <izzy@qumran.org>
* JackDesBwa <jackdesbwa@gmail.com>
* Jannes Blobel <72493222+jannesblobel@users.noreply.github.com>
* Jeffery Wilkins <goaccess@prosoftcorp.com>
* Jeremy Burks <jeremy@zomgwat.com>
* Jeremy Lin <jeremy.lin@gmail.com>
* Joaquín de la Zerda <joaquin@itbaf.com>
* Joe Groocock <me@frebib.net>
* Joe Winett <noreply@github.com>
* Jonas Kittner <jonas.kittner@rub.de>
* Joona <joona.paivahonka@gmail.com>
* Jordan Trask <jtrask@gmail.com>
* Josh Briggs <josh@briggsdev.us>
* Julian Xhokaxhiu <xhokaxhiujulian@gmail.com>
* Justin Mills <justin.mills@utah.edu>
* Kamino Hiroki <37243867+4f8p@users.noreply.github.com>
* Kit Westneat <kit.westneat@gmail.com>
* Koichiro Iwao <meta@vmeta.jp>
* kokke <spam@rowdy.dk>
* kyle sloan <kyle@ifsight.com>
* LeoAttn <leo.massonnat@hotmail.fr>
* lhywk <lhywkd22@gmail.com>
* Linus Gasser <linus.gasser@epfl.ch>
* Magnus Groß <magnus@mggross.com>
* Maksim Losev <mlosev@beget.ru>
* mario-donnarumma <mario.donnarumma.97@gmail.com>
* markiewb <noreply@github.com>
* Mark J. Berger <mark.berger.j@gmail.com>
* Martins Polakovs <martins.polakovs@gmail.com>
* Massimiliano Torromeo <massimiliano.torromeo@gmail.com>
* Mathieu Aubin <noreply@github.com>
* Mathieu Thoretton <mathieu.thoretton@getyourguide.com>
* Max Christian Pohle <goaccess@prosoftcorp.com>
* metrix78 <metrix1978@gmail.com>
* Michael Vetter <jubalh@iodoru.org>
* Mika Raunio <mika@diago.global>
* Moritz Schott <unsafe@nurfuerspam.de>
* m-r-r <raybaudroigm@gmail.com>
* mynameiscfed <christopherfederico@gmail.com>
* Newbe36524 <newbe36524@qq.com>
* Nicolas Le Manchet <nicolas@lemanchet.fr>
* Nicolas <noreply@github.com>
* Ophir LOJKINE <pere.jobs@gmail.com>
* Otto Kekäläinen <otto.kekalainen@seravo.fi>
* Panos Stavrianos <panos-stavrianos@users.noreply.github.com>
* pitilux <pitilux@elinar.fr>
* Pixelcode <52963327+realpixelcode@users.noreply.github.com>
* Placidina <alanplacidina@gmail.com>
* pravdomil <noreply@github.com>
* rachid-debu <rachid.debu@yahoo.fr>
* radoslawc <radoslawc@gmail.com>
* rahra <bf@abenteuerland.at>
* Ramires Viana <59319979+ramiresviana@users.noreply.github.com>
* rgriebl <robert@griebl.org>
* Roy Marples <roy@marples.name>
* rtmkrlv <artyom.karlov@gmail.com>
* Ryow <ryow@midgen.asia>
* schoonc <noreply@github.com>
* Sean Cross <sean@xobs.io>
* Sean Wei <me@sean.taipei>
* Sebastian Wiedenroth <wiedi@frubar.net>
* sh4rkman <maxime.boussard@gmail.com>
* Simon Gardling <Titaniumtown@gmail.com>
* SjonHortensius <noreply@github.com>
* Steely Wing <noreply@github.com>
* Stéphane Péchard <stephanepechard@gmail.com>
* Stephen Wade <stephen@stephenwade.me>
* Stoyan Dimov <s.dimov@codelogic.eu>
* Stuart Henderson <noreply@github.com>
* Sveinbjorn Thordarson <sveinbjorn@sveinbjorn.org>
* Tatsuyuki Ishi <noreply@github.com>
* Thomas Gläßle <t_glaessle@gmx.de>
* Thomas Jost <schnouki@schnouki.net>
* Thomas Lange <code@nerdmind.de>
* throwaway1037 <noreply@github.com>
* Tim Gates <tim.gates@iress.com>
* Timothy Quilling <teqed@shatteredsky.net>
* Tom Samstag <github@modtwo.com>
* ugola <urvika.gola@gmail.com>
* Ulrich Schwarz <ulmi@absatzen.de>
* Viktor Szépe <noreply@github.com>
* Viktor Szépe <viktor@szepe.net>
* Ville Skyttä <ville.skytta@iki.fi>
* Vincent Bernat <vincent@bernat.ch>
* Vladimir Pavljuchenkov <spiderx@spiderx.dp.ua>
* William Muir <wamuir@gmail.com>
* Wladimir Palant <noreply@github.com>
* wodev <wodev@fischer-net.net>
* woobee <support.developpement@gmail.com>
* Yaroslav Halchenko <debian@onerussian.com>
* Yuri D'Elia <wavexx@thregr.org>
* Yuriy M. Kaminskiy <yumkam@gmail.com>
* zeke <zhenkai0605@gmail.com>
COPYING
+21
View File
@@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2009-2022 Gerardo Orellana <hello @ goaccess.io>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
ChangeLog
+1263
View File
File diff suppressed because it is too large Load Diff
DOCKER.md
+31
View File
@@ -0,0 +1,31 @@
# Docker image features
* This command uses the language set for this system. If that does not support it will be output in English. [**Supported Language**](https://github.com/allinurl/goaccess/raw/master/po/LINGUAS)
* This image supports building on the ARM architecture (e.g. Raspberry Pi)
* Do you want to change the timezone? Use the `-e` option to pass the time-zone setting to Docker. (e.g. `-e TZ="America/New_York"`)
* The container is built with geo-location support (see [the manual](https://goaccess.io/man#options)). To enable the respective panel, mount the geolocation database using `-v /path/to/GeoLite2-City.mmdb:/GeoLite2-City.mmdb` and specify `--geoip-database /GeoLite2-City.mmdb` when running GoAccess.
* If you made changes to the config file after building the image, you don't have to rebuild from scratch. Simply restart the container:
```
docker restart goaccess
```
* If you had already run the container, you may have to stop and remove it first:
```
docker stop goaccess
docker rm goaccess
```
* The container and image can be completely removed as follows:
```
docker stop goaccess
docker rm goaccess
docker rmi allinurl/goaccess
```
Dockerfile
+41
View File
@@ -0,0 +1,41 @@
# Build stage
FROM alpine:3.20 AS builds
RUN apk add --no-cache \
autoconf \
automake \
build-base \
clang \
gettext-dev \
libmaxminddb-dev \
openssl-dev \
linux-headers \
ncurses-dev \
pkgconf \
tzdata
# GoAccess
COPY . /goaccess
WORKDIR /goaccess
RUN autoreconf -fiv && rm -rf autom4te.cache
RUN CC="clang" CFLAGS="-O3" LIBS="$(pkg-config --libs openssl)" ./configure --prefix=/usr --enable-utf8 --with-openssl --enable-geoip=mmdb
RUN make -j$(nproc) && make DESTDIR=/dist install
# Check dynamic dependencies
RUN ldd /dist/usr/bin/goaccess && echo "Dependencies checked"
# Runtime stage
FROM alpine:3.20
RUN apk add --no-cache \
gettext-libs \
libmaxminddb \
ncurses-libs \
openssl \
tzdata
# Copy GoAccess binary and assets
COPY --from=builds /dist/usr/bin/goaccess /usr/bin/goaccess
COPY --from=builds /dist/usr/share /usr/share
COPY --from=builds /usr/share/zoneinfo /usr/share/zoneinfo
# Set up volume and port
VOLUME /var/www/goaccess
EXPOSE 7890
ENTRYPOINT ["/usr/bin/goaccess"]
CMD ["--help"]
Dockerfile.debian-12
+17
View File
@@ -0,0 +1,17 @@
# Used to have all compile dependencies isolated in a container image.
FROM debian:12
RUN apt update -qqq
RUN apt install -y \
libncurses-dev \
libssl-dev \
libmaxminddb-dev \
build-essential \
autoconf \
gettext \
autopoint
# GoAccess
WORKDIR /goaccess
ENTRYPOINT ["./build-dynamic.sh"]
Makefile.am
+292
View File
@@ -0,0 +1,292 @@
#AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = goaccess
AUTOMAKE_OPTIONS = subdir-objects
dist_noinst_DATA = \
resources/tpls.html \
resources/css/app.css \
resources/css/bootstrap.min.css \
resources/css/fa.min.css \
resources/js/app.js \
resources/js/charts.js \
resources/countries-110m.json \
resources/cities-10m.json \
resources/js/d3.v7.min.js \
resources/js/topojson.v3.min.js \
resources/js/hogan.min.js
noinst_PROGRAMS = bin2c
bin2c_SOURCES = src/bin2c.c
BUILT_SOURCES = \
src/tpls.h \
src/bootstrapcss.h \
src/facss.h \
src/appcss.h \
src/d3js.h \
src/topojsonjs.h \
src/hoganjs.h \
src/countries110m.h \
src/cities10m.h \
src/chartsjs.h \
src/appjs.h
CLEANFILES = \
src/tpls.h \
src/bootstrapcss.h \
src/facss.h \
src/appcss.h \
src/d3js.h \
src/topojsonjs.h \
src/hoganjs.h \
src/countries110m.h \
src/cities10m.h \
src/chartsjs.h \
src/appjs.h \
resources/tpls.html.tmp \
resources/countries-110m.json.tmp \
resources/cities-10m.json.tmp \
resources/css/bootstrap.min.css.tmp \
resources/css/fa.min.css.tmp \
resources/css/app.css.tmp \
resources/js/d3.v7.min.js.tmp \
resources/js/topojson.v3.min.js.tmp \
resources/js/hogan.min.js.tmp \
resources/js/charts.js.tmp \
resources/js/app.js.tmp
# Tpls
src/tpls.h: bin2c$(EXEEXT) $(srcdir)/resources/tpls.html
if HAS_SEDTR
cat $(srcdir)/resources/tpls.html | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/tpls.html.tmp
./bin2c $(srcdir)/resources/tpls.html.tmp src/tpls.h tpls
else
./bin2c $(srcdir)/resources/tpls.html src/tpls.h tpls
endif
# countries.json
src/countries110m.h: bin2c$(EXEEXT) $(srcdir)/resources/countries-110m.json
if HAS_SEDTR
cat $(srcdir)/resources/countries-110m.json | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/countries-110m.json.tmp
./bin2c $(srcdir)/resources/countries-110m.json.tmp src/countries110m.h countries_json
else
./bin2c $(srcdir)/resources/countries-110m.json src/countries110m.h countries_json
endif
# cities.json
src/cities10m.h: bin2c$(EXEEXT) $(srcdir)/resources/cities-10m.json
if HAS_SEDTR
cat $(srcdir)/resources/cities-10m.json | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/cities-10m.json.tmp
./bin2c $(srcdir)/resources/cities-10m.json.tmp src/cities10m.h cities_json
else
./bin2c $(srcdir)/resources/cities-10m.json src/cities10m.h cities_json
endif
# Bootstrap
src/bootstrapcss.h: bin2c$(EXEEXT) $(srcdir)/resources/css/bootstrap.min.css
if HAS_SEDTR
cat $(srcdir)/resources/css/bootstrap.min.css | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/css/bootstrap.min.css.tmp
./bin2c $(srcdir)/resources/css/bootstrap.min.css.tmp src/bootstrapcss.h bootstrap_css
else
./bin2c $(srcdir)/resources/css/bootstrap.min.css src/bootstrapcss.h bootstrap_css
endif
# Font Awesome
src/facss.h: bin2c$(EXEEXT) $(srcdir)/resources/css/fa.min.css
if HAS_SEDTR
cat $(srcdir)/resources/css/fa.min.css | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/css/fa.min.css.tmp
./bin2c $(srcdir)/resources/css/fa.min.css.tmp src/facss.h fa_css
else
./bin2c $(srcdir)/resources/css/fa.min.css src/facss.h fa_css
endif
# App.css
src/appcss.h: bin2c$(EXEEXT) $(srcdir)/resources/css/app.css
if HAS_SEDTR
cat $(srcdir)/resources/css/app.css | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/css/app.css.tmp
./bin2c $(srcdir)/resources/css/app.css.tmp src/appcss.h app_css
else
./bin2c $(srcdir)/resources/css/app.css src/appcss.h app_css
endif
# D3.js
src/d3js.h: bin2c$(EXEEXT) $(srcdir)/resources/js/d3.v7.min.js
if HAS_SEDTR
cat $(srcdir)/resources/js/d3.v7.min.js | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/js/d3.v7.min.js.tmp
./bin2c $(srcdir)/resources/js/d3.v7.min.js.tmp src/d3js.h d3_js
else
./bin2c $(srcdir)/resources/js/d3.v7.min.js src/d3js.h d3_js
endif
# topojson.js
src/topojsonjs.h: bin2c$(EXEEXT) $(srcdir)/resources/js/topojson.v3.min.js
if HAS_SEDTR
cat $(srcdir)/resources/js/topojson.v3.min.js | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/js/topojson.v3.min.js.tmp
./bin2c $(srcdir)/resources/js/topojson.v3.min.js.tmp src/topojsonjs.h topojson_js
else
./bin2c $(srcdir)/resources/js/topojson.v3.min.js src/topojsonjs.h topojson_js
endif
# Hogan.js
src/hoganjs.h: bin2c$(EXEEXT) $(srcdir)/resources/js/hogan.min.js
if HAS_SEDTR
cat $(srcdir)/resources/js/hogan.min.js | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/js/hogan.min.js.tmp
./bin2c $(srcdir)/resources/js/hogan.min.js.tmp src/hoganjs.h hogan_js
else
./bin2c $(srcdir)/resources/js/hogan.min.js src/hoganjs.h hogan_js
endif
# Charts.js
src/chartsjs.h: bin2c$(EXEEXT) $(srcdir)/resources/js/charts.js
if HAS_SEDTR
cat $(srcdir)/resources/js/charts.js | sed -E "s@(,|;)[[:space:]]*//..*@\1@g" | sed -E "s@^[[:space:]]*//..*@@g" | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/js/charts.js.tmp
./bin2c $(srcdir)/resources/js/charts.js.tmp src/chartsjs.h charts_js
else
./bin2c $(srcdir)/resources/js/charts.js src/chartsjs.h charts_js
endif
if DEBUG
./bin2c $(srcdir)/resources/js/charts.js src/chartsjs.h charts_js
endif
# App.js
src/appjs.h: bin2c$(EXEEXT) $(srcdir)/resources/js/app.js
if HAS_SEDTR
cat $(srcdir)/resources/js/app.js | sed -E "s@(,|;)[[:space:]]*//..*@\1@g" | sed -E "s@^[[:space:]]*//..*@@g" | sed "s/^[[:space:]]*//" | sed "/^$$/d" | tr -d "\r\n" > $(srcdir)/resources/js/app.js.tmp
./bin2c $(srcdir)/resources/js/app.js.tmp src/appjs.h app_js
else
./bin2c $(srcdir)/resources/js/app.js src/appjs.h app_js
endif
if DEBUG
./bin2c $(srcdir)/resources/js/app.js src/appjs.h app_js
endif
confdir = $(sysconfdir)/goaccess
dist_conf_DATA = config/goaccess.conf
dist_conf_DATA += config/browsers.list
dist_conf_DATA += config/podcast.list
goaccess_SOURCES = \
src/base64.c \
src/base64.h \
src/browsers.c \
src/browsers.h \
src/color.c \
src/color.h \
src/commons.c \
src/commons.h \
src/csv.c \
src/csv.h \
src/dialogs.c \
src/dialogs.h \
src/error.c \
src/error.h \
src/fileio.c \
src/fileio.h \
src/gchart.c \
src/gchart.h \
src/gdashboard.c \
src/gdashboard.h \
src/gdns.c \
src/gdns.h \
src/gholder.c \
src/gholder.h \
src/gkhash.c \
src/gkhash.h \
src/gkmhash.c \
src/gkmhash.h \
src/gmenu.c \
src/gmenu.h \
src/goaccess.c \
src/goaccess.h \
src/gslist.c \
src/gslist.h \
src/gstorage.c \
src/gstorage.h \
src/gwsocket.c \
src/gwsocket.h \
src/json.c \
src/json.h \
src/khash.h \
src/labels.h \
src/opesys.c \
src/opesys.h \
src/options.c \
src/options.h \
src/output.c \
src/output.h \
src/parser.c \
src/parser.h \
src/persistence.c \
src/persistence.h \
src/pdjson.c \
src/pdjson.h \
src/settings.c \
src/settings.h \
src/sort.c \
src/sort.h \
src/tpl.c \
src/tpl.h \
src/ui.c \
src/ui.h \
src/util.c \
src/util.h \
src/websocket.c \
src/websocket.h \
src/xmalloc.c \
src/xmalloc.h
if WITH_SSL
goaccess_SOURCES += \
src/wsauth.c \
src/wsauth.h
endif
if USE_SHA1
goaccess_SOURCES += \
src/sha1.c \
src/sha1.h
endif
if USE_MMAP
goaccess_SOURCES += \
src/win/mman.h \
src/win/mmap.c
endif
if GEOIP_LEGACY
goaccess_SOURCES += \
src/geoip1.c \
src/geoip1.h
endif
if GEOIP_MMDB
goaccess_SOURCES += \
src/geoip2.c \
src/geoip1.h
endif
if DEBUG
AM_CFLAGS = -DDEBUG -O0 -DSYSCONFDIR=\"$(sysconfdir)\"
else
AM_CFLAGS = -O2 -DSYSCONFDIR=\"$(sysconfdir)\"
endif
if WITH_RDYNAMIC
AM_LDFLAGS = -rdynamic
endif
if WITH_ZLIB
AM_LDFLAGS = -lz
endif
goaccess_LDADD = -lm
AM_CFLAGS += -Wall -Wextra -Wnested-externs -Wformat=2 -g
AM_CFLAGS += -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations
AM_CFLAGS += -Wwrite-strings -Wshadow -Wpointer-arith -Wsign-compare
AM_CFLAGS += -Wbad-function-cast -Wcast-align
AM_CFLAGS += -Wdeclaration-after-statement -Wshadow -Wold-style-definition
if WITH_ASAN
AM_CFLAGS += -fsanitize=address
endif
dist_man_MANS = goaccess.1
SUBDIRS = po
ACLOCAL_AMFLAGS = -I m4
DEFS = -DLOCALEDIR=\"$(localedir)\" @DEFS@
EXTRA_DIST = config.rpath
NEWS
+148
View File
@@ -0,0 +1,148 @@
Copyright (C) 2009-2026
Gerardo Orellana <goaccess@prosoftcorp.com>
* Version history:
- 1.10.2 [Tuesday, March 31, 2026]
. GoAccess 1.10.2 Released. See ChangeLog for new features/bug-fixes.
- 1.10.1 [Saturday, Feb 14, 2026]
. GoAccess 1.10.1 Released. See ChangeLog for new features/bug-fixes.
- 1.10 [Tuesday, Feb 10, 2026]
. GoAccess 1.10 Released. See ChangeLog for new features/bug-fixes.
- 1.9.4 [Monday, Mar 31, 2025]
. GoAccess 1.9.4 Released. See ChangeLog for new features/bug-fixes.
- 1.9.3 [Friday, May 31, 2024]
. GoAccess 1.9.3 Released. See ChangeLog for new features/bug-fixes.
- 1.9.2 [Friday, April 12, 2024]
. GoAccess 1.9.2 Released. See ChangeLog for new features/bug-fixes.
- 1.9.1 [Tuesday, February 05, 2024]
. GoAccess 1.9.1 Released. See ChangeLog for new features/bug-fixes.
- 1.9 [Tuesday, January 30, 2024]
. GoAccess 1.9 Released. See ChangeLog for new features/bug-fixes.
- 1.8.1 [Tuesday, October 31, 2023]
. GoAccess 1.8.1 Released. See ChangeLog for new features/bug-fixes.
- 1.8 [Saturday, September 30, 2023]
. GoAccess 1.8 Released. See ChangeLog for new features/bug-fixes.
- 1.7.2 [Friday, March 31, 2023]
. GoAccess 1.7.2 Released. See ChangeLog for new features/bug-fixes.
- 1.7.1 [Tuesday, February 28, 2023]
. GoAccess 1.7.1 Released. See ChangeLog for new features/bug-fixes.
- 1.7 [Saturday, December 31 , 2022]
. GoAccess 1.7 Released. See ChangeLog for new features/bug-fixes.
- 1.6.5 [Monday, October 31 , 2022]
. GoAccess 1.6.5 Released. See ChangeLog for new features/bug-fixes.
- 1.6.4 [Friday, September 30 , 2022]
. GoAccess 1.6.4 Released. See ChangeLog for new features/bug-fixes.
- 1.6.3 [Thursday, August 31 , 2022]
. GoAccess 1.6.3 Released. See ChangeLog for new features/bug-fixes.
- 1.6.2 [Thursday, July 14 , 2022]
. GoAccess 1.6.2 Released. See ChangeLog for new features/bug-fixes.
- 1.6.1 [Thursday, June 30 , 2022]
. GoAccess 1.6.1 Released. See ChangeLog for new features/bug-fixes.
- 1.6 [Tuesday, May 31 , 2022]
. GoAccess 1.6 Released. See ChangeLog for new features/bug-fixes.
- 1.5.7 [Thursday, April 28 , 2022]
. GoAccess 1.5.7 Released. See ChangeLog for new features/bug-fixes.
- 1.5.6 [Wednesday, March 30, 2022]
. GoAccess 1.5.6 Released. See ChangeLog for new features/bug-fixes.
- 1.5.5 [Monday, January 31, 2022]
. GoAccess 1.5.5 Released. See ChangeLog for new features/bug-fixes.
- 1.5.4 [Saturday, December 25, 2021]
. GoAccess 1.5.4 Released. See ChangeLog for new features/bug-fixes.
- 1.5.3 [Thursday, November 25, 2021]
. GoAccess 1.5.3 Released. See ChangeLog for new features/bug-fixes.
- 1.5.2 [Tuesday, Sep 28, 2021]
. GoAccess 1.5.2 Released. See ChangeLog for new features/bug-fixes.
- 1.5.1 [Wednesday, Jun 30, 2021]
. GoAccess 1.5.1 Released. See ChangeLog for new features/bug-fixes.
- 1.5 [Wednesday, May 26, 2021]
. GoAccess 1.5 Released. See ChangeLog for new features/bug-fixes.
- 1.4.6 [Sunday, February 28, 2021]
. GoAccess 1.4.6 Released. See ChangeLog for new features/bug-fixes.
- 1.4.5 [Tuesday, January 26, 2021]
. GoAccess 1.4.5 Released. See ChangeLog for new features/bug-fixes.
- 1.4.4 [Monday, January 25, 2021]
. GoAccess 1.4.4 Released. See ChangeLog for new features/bug-fixes.
- 1.4.3 [Friday, December 04, 2020]
. GoAccess 1.4.3 Released. See ChangeLog for new features/bug-fixes.
- 1.4.2 [Monday, November 16, 2020]
. GoAccess 1.4.2 Released. See ChangeLog for new features/bug-fixes.
- 1.4.1 [Monday, November 09, 2020]
. GoAccess 1.4.1 Released. See ChangeLog for new features/bug-fixes.
- 1.4 [Monday, May 18, 2020]
. GoAccess 1.4 Released. See ChangeLog for new features/bug-fixes.
- 1.3 [Friday, November 23, 2018]
. GoAccess 1.3 Released. See ChangeLog for new features/bug-fixes.
- 1.2 [Tuesday, March 07, 2017]
. GoAccess 1.2 Released. See ChangeLog for new features/bug-fixes.
- 1.1.1 [Wednesday, November 23, 2016]
. GoAccess 1.1.1 Released. See ChangeLog for new features/bug-fixes.
- 1.1 [Tuesday, November 08, 2016]
. GoAccess 1.1 Released. See ChangeLog for new features/bug-fixes.
- 1.0.2 [Tuesday, July 05, 2016]
. GoAccess 1.0.2 Released. See ChangeLog for new features/bug-fixes.
- 1.0.1 [Friday, June 17, 2016]
. GoAccess 1.0.1 Released. See ChangeLog for new features/bug-fixes.
- 1.0 [Thursday, June 09, 2016]
. GoAccess 1.0 Released. See ChangeLog for new features/bug-fixes.
- 0.9.8 [Monday, February 29, 2016]
. GoAccess 0.9.8 Released. See ChangeLog for new features/bug-fixes.
- 0.9.7 [Monday, December 21, 2015]
. GoAccess 0.9.7 Released. See ChangeLog for new features/bug-fixes.
- 0.9.6 [Tuesday, October 27, 2015]
. GoAccess 0.9.6 Released. See ChangeLog for new features/bug-fixes.
- 0.9.5 [Thursday, October 22, 2015]
. GoAccess 0.9.5 Released. See ChangeLog for new features/bug-fixes.
- 0.9.4 [Tuesday, September 08 , 2015]
. GoAccess 0.9.4 Released. See ChangeLog for new features/bug-fixes.
- 0.9.3 [Wednesday, August 28, 2015]
. GoAccess 0.9.3 Released. See ChangeLog for new features/bug-fixes.
- 0.9.2 [Monday, July 06, 2015]
. GoAccess 0.9.2 Released. See ChangeLog for new features/bug-fixes.
- 0.9.1 [Tuesday, May 26, 2015]
. GoAccess 0.9.1 Released. See ChangeLog for new features/bug-fixes.
- 0.9 [Thursday, March 19, 2015]
. GoAccess 0.9 Released. See ChangeLog for new features/bug-fixes.
- 0.8.5 [Sunday, September 14, 2014]
. GoAccess 0.8.5 Released. See ChangeLog for new features/bug-fixes.
- 0.8.4 [Monday, September 08, 2014]
. GoAccess 0.8.4 Released. See ChangeLog for new features/bug-fixes.
- 0.8.3 [Monday, July 28, 2014]
. GoAccess 0.8.3 Released. See ChangeLog for new features/bug-fixes.
- 0.8.2 [Monday, July 21, 2014]
. GoAccess 0.8.2 Released. See ChangeLog for new features/bug-fixes.
- 0.8.1 [Monday, June 16, 2014]
. GoAccess 0.8.1 Released. See ChangeLog for new features/bug-fixes.
- 0.8 [Monday, May 20, 2013]
. GoAccess 0.8 Released. See ChangeLog for new features/bug-fixes.
- 0.7.1 [Monday, February 17, 2014]
. GoAccess 0.7.1 Released. See ChangeLog for new features/bug-fixes.
- 0.7 [Monday, December 16, 2013]
. GoAccess 0.7 Released. See ChangeLog for new features/bug-fixes.
- 0.6.1 [Monday, October 07, 2013]
. GoAccess 0.6.1 Released. See ChangeLog for new features/bug-fixes.
- 0.6 [Monday, July 15, 2013]
. GoAccess 0.6 Released. See ChangeLog for new features/bug-fixes.
- 0.5 [Monday, June 04, 2012]
. GoAccess 0.5 Released. See ChangeLog for new features/bug-fixes.
- 0.4.2 [Monday, January 03, 2011]
. GoAccess 0.4.2 Released. See ChangeLog for new features/bug-fixes.
- 0.4.1 [Monday, December 13, 2010]
. GoAccess 0.4.1 Released. See ChangeLog for new features/bug-fixes.
- 0.4 [Tuesday, November 30, 2010]
. GoAccess 0.4 Released. See ChangeLog for new features/bug-fixes.
- 0.3.3 [Monday, September 27, 2010]
. GoAccess 0.3.3 Released. See ChangeLog for new features/bug-fixes.
- 0.3.2 [Thursday, September 09 2010]
. GoAccess 0.3.2 Released. See ChangeLog for new features/bug-fixes.
- 0.3.1 [Friday, September 03, 2010]
. GoAccess 0.3.1 Released. See ChangeLog for new features/bug-fixes.
- 0.3 [Sunday, August 29, 2010]
. GoAccess 0.3 Released. See ChangeLog for new features/bug-fixes.
- 0.2 [Sunday, July 25, 2010]
. GoAccess 0.2 Released. See ChangeLog for new features/bug-fixes.
- 0.1.2 [Tuesday, July 13, 2010]
. GoAccess 0.1.2 Released. See ChangeLog for new features/bug-fixes.
- 0.1.1 [Saturday, July 10, 2010]
. GoAccess 0.1.1 Released. See ChangeLog for new features/bug-fixes.
- 0.1 [Wednesday, July 07, 2010]
. Welcome to the GoAccess 0.1 Released.
README.md
+514
View File
@@ -0,0 +1,514 @@
GoAccess [![C build](https://github.com/allinurl/goaccess/actions/workflows/build-test.yml/badge.svg)](https://github.com/allinurl/goaccess/actions/workflows/build-test.yml) [![GoAccess](https://goaccess.io/badge)](https://goaccess.io)
========
## What is it? ##
GoAccess is an open source, real-time web log analyzer and interactive viewer
that runs in a terminal on *nix systems or directly in your browser. Designed
with system administrators, DevOps engineers, and security professionals in
mind, it delivers fast, actionable HTTP statistics and visual server reports on
the fly. GoAccess parses your web server logs in real time and presents the
data directly in the terminal or via a live HTML dashboard, making it easy to
monitor traffic, detect anomalies, and troubleshoot issues instantly.
More info at: [https://goaccess.io](https://goaccess.io/?src=gh).
[![GoAccess Terminal Dashboard](https://goaccess.io/images/goaccess-real-time-term-gh-2026-1.png?2026021201)](https://goaccess.io/)
[![GoAccess HTML Dashboard](https://goaccess.io/images/goaccess-real-time-html-gh-2026.png?2026021201)](https://rt.goaccess.io/?src=gh)
## Features ##
GoAccess parses the specified web log file and outputs the data to the X
terminal. Features include:
* **Completely Real Time**<br>
All panels and metrics are timed to be updated every 200 ms on the terminal
output and every second on the HTML output.
* **Minimal Configuration needed**<br>
You can just run it against your access log file, pick the log format and let
GoAccess parse the access log and show you the stats.
* **Track Application Response Time**<br>
Track the time taken to serve the request. Extremely useful if you want to
track pages that are slowing down your site.
* **WebSocket Authentication:**<br>
GoAccess offers enhanced WebSocket authentication, supporting local and
external JWT verification, with secure token refresh capabilities and seamless
integration with external authentication systems.
* **Nearly All Web Log Formats**<br>
GoAccess allows any custom log format string. Predefined options include,
Apache, Nginx, Amazon S3, Elastic Load Balancing, CloudFront, etc.
* **Incremental Log Processing**<br>
Need data persistence? GoAccess has the ability to process logs incrementally
through the on-disk persistence options.
* **Only one dependency**<br>
GoAccess is written in C. To run it, you only need ncurses as a dependency.
That's it. It even features its own Web Socket server — http://gwsocket.io/.
* **Visitors**<br>
Determine the amount of hits, visitors, bandwidth, and metrics for slowest
running requests by the hour, or date.
* **Metrics per Virtual Host**<br>
Have multiple Virtual Hosts (Server Blocks)? It features a panel that
displays which virtual host is consuming most of the web server resources.
* **ASN (Autonomous System Number mapping)**<br>
Great for detecting malicious traffic patterns and block them accordingly.
* **Color Scheme Customizable**<br>
Tailor GoAccess to suit your own color taste/schemes. Either through the
terminal, or by simply applying the stylesheet on the HTML output.
* **Support for Large Datasets**<br>
GoAccess features the ability to parse large logs due to its optimized
in-memory hash tables. It has very good memory usage and pretty good
performance. This storage has support for on-disk persistence as well.
* **Docker Support**<br>
Ability to build GoAccess' Docker image from upstream. You can still fully
configure it, by using Volume mapping and editing `goaccess.conf`. See
[Docker](https://github.com/allinurl/goaccess#docker) section below.
There is also documentation how to use [docker-compose](./docker-compose/README.md).
### Nearly all web log formats... ###
GoAccess allows any custom log format string. Predefined options include, but
not limited to:
* Amazon CloudFront (Download Distribution).
* Amazon Simple Storage Service (S3)
* AWS Elastic Load Balancing
* Combined Log Format (XLF/ELF) Apache | Nginx
* Common Log Format (CLF) Apache
* Google Cloud Storage.
* Apache virtual hosts
* Squid Native Format.
* W3C format (IIS).
* Caddy's JSON Structured format.
* Traefik's CLF flavor
## Why GoAccess? ##
GoAccess was designed to be a fast, terminal-based log analyzer. Its core idea
is to quickly analyze and view web server statistics in real time without
needing to use your browser (_great if you want to do a quick analysis of your
access log via SSH, or if you simply love working in the terminal_).
It also serves as a practical tool for security monitoring, making it easy to
spot suspicious activity, unusual traffic patterns, brute-force attempts,
scanners, bots, and anomalous requests directly from your logs.
While the terminal output is the default output, it has the capability to
generate a complete, self-contained, real-time [**`HTML`**](https://rt.goaccess.io/?src=gh)
report, as well as a [**`JSON`**](https://goaccess.io/json?src=gh), and
[**`CSV`**](https://goaccess.io/goaccess_csv_report.csv?src=gh) report.
You can see it more of a monitor command tool than anything else.
## Installation ##
<a href="https://repology.org/project/goaccess/versions">
<img src="https://repology.org/badge/vertical-allrepos/goaccess.svg" alt="Packaging status" align="right">
</a>
### Build from release
GoAccess can be compiled and used on *nix systems.
Download, extract and compile GoAccess with:
$ wget https://tar.goaccess.io/goaccess-1.10.2.tar.gz
$ tar -xzvf goaccess-1.10.2.tar.gz
$ cd goaccess-1.10.2/
$ ./configure --enable-utf8 --enable-geoip=mmdb --with-zlib
$ make
# make install
### Build from GitHub (Development) ###
$ git clone https://github.com/allinurl/goaccess.git
$ cd goaccess
$ autoreconf -fiv
$ ./configure --enable-utf8 --enable-geoip=mmdb
$ make
# make install
### Distributions ###
It is easiest to install GoAccess on GNU+Linux using the preferred package manager
of your GNU+Linux distribution. Please note that not all distributions will have
the latest version of GoAccess available.
#### Debian/Ubuntu ####
# apt-get install goaccess
**Note:** It is likely this will install an outdated version of GoAccess. To
make sure that you're running the latest stable version of GoAccess see
alternative option below.
#### Official GoAccess Debian & Ubuntu repository ####
$ wget -O - https://deb.goaccess.io/gnugpg.key | gpg --dearmor | sudo tee /usr/share/keyrings/goaccess.gpg >/dev/null
$ echo "deb [signed-by=/usr/share/keyrings/goaccess.gpg arch=$(dpkg --print-architecture)] https://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/goaccess.list
$ sudo apt-get update
$ sudo apt-get install goaccess
**Note**:
* `.deb` packages in the official repo are available through HTTPS as well. You may need to install `apt-transport-https`.
#### Fedora ####
# yum install goaccess
#### Arch ####
# pacman -S goaccess
#### Gentoo ####
# emerge net-analyzer/goaccess
#### OS X / Homebrew ####
# brew install goaccess
#### FreeBSD ####
# cd /usr/ports/sysutils/goaccess/ && make install clean
# pkg install sysutils/goaccess
#### OpenBSD ####
# cd /usr/ports/www/goaccess && make install clean
# pkg_add goaccess
#### openSUSE ####
# zypper ar -f obs://server:http http
# zypper in goaccess
#### OpenIndiana ####
# pkg install goaccess
#### pkgsrc (NetBSD, Solaris, SmartOS, ...) ####
# pkgin install goaccess
#### Windows ####
GoAccess can be used in Windows through Cygwin. See Cygwin's <a
href="https://goaccess.io/faq#installation">packages</a>. Or through the
GNU+Linux Subsystem on Windows 10.
#### Docker ####
A Docker image has been updated, capable of directing output from an access log. If you only want to output a report, you can pipe a log from the external environment to a Docker-based process:
touch report.html
cat access.log | docker run --rm -i -v ./report.html:/report.html -e LANG=$LANG allinurl/goaccess -a -o report.html --log-format COMBINED -
OR real-time
tail -F access.log | docker run -p 7890:7890 --rm -i -e LANG=$LANG allinurl/goaccess -a -o report.html --log-format COMBINED --real-time-html -
There is also documentation how to use [docker-compose](./docker-compose/README.md).
##### Build in isolated container
You can also build the binary for Debian based systems in an isolated container environment to prevent cluttering your local system with the development libraries:
$ curl -L "https://github.com/allinurl/goaccess/archive/refs/heads/master.tar.gz" | tar -xz && cd goaccess-master
$ docker build -t goaccess/build.debian-12 -f Dockerfile.debian-12 .
$ docker run -i --rm -v $PWD:/goaccess goaccess/build.debian-12 > goaccess
You can read more about using the docker image in [DOCKER.md](https://github.com/allinurl/goaccess/blob/master/DOCKER.md).
#### Distribution Packages ####
GoAccess has minimal requirements, it's written in C and requires only ncurses.
However, below is a table of some optional dependencies in some distros to
build GoAccess from source.
| Distro | NCurses | GeoIP (opt) | GeoIP2 (opt) | OpenSSL (opt) |
| ---------------------- | ---------------- | ---------------- | --------------------- | -------------------|
| **Ubuntu/Debian** | libncurses-dev | libgeoip-dev | libmaxminddb-dev | libssl-dev |
| **RHEL/CentOS** | ncurses-devel | geoip-devel | libmaxminddb-devel | openssl-devel |
| **Arch** | ncurses | geoip | libmaxminddb | openssl |
| **Gentoo** | sys-libs/ncurses | dev-libs/geoip | dev-libs/libmaxminddb | dev-libs/openssl |
| **Slackware** | ncurses | GeoIP | libmaxminddb | openssl |
**Note**: You may need to install build tools like `gcc`, `autoconf`,
`gettext`, `autopoint` etc. for compiling/building software from source. e.g.,
`base-devel`, `build-essential`, `"Development Tools"`.
## Storage ##
#### Default Hash Tables ####
In-memory storage provides better performance at the cost of limiting the
dataset size to the amount of available physical memory. GoAccess uses
in-memory hash tables. It has very good memory usage and pretty good
performance. This storage has support for on-disk persistence as well.
## Command Line / Config Options ##
See [**options**](https://goaccess.io/man#options) that can be supplied to the command or
specified in the configuration file. If specified in the configuration file, long
options need to be used without prepending `--`.
## Usage / Examples ##
**Note**: Piping data into GoAccess won't prompt a log/date/time
configuration dialog, you will need to previously define it in your
configuration file or in the command line.
### Getting Started ###
To output to a terminal and generate an interactive report:
# goaccess access.log
To generate an HTML report:
# goaccess access.log -a > report.html
To generate a JSON report file:
# goaccess access.log -a -d -o report.json
To generate a CSV report to stdout:
# goaccess access.log --no-csv-summary -o csv
GoAccess also allows great flexibility for real-time filtering and parsing. For
instance, to quickly diagnose issues by monitoring logs since goaccess was
started:
# tail -f access.log | goaccess -
And even better, to filter while maintaining opened a pipe to preserve
real-time analysis, we can make use of `tail -f` and a matching pattern tool
such as `grep`, `awk`, `sed`, etc.:
# tail -f access.log | grep -i --line-buffered 'firefox' | goaccess --log-format=COMBINED -
or to parse from the beginning of the file while maintaining the pipe opened
and applying a filter
# tail -f -n +0 access.log | grep -i --line-buffered 'firefox' | goaccess -o report.html --real-time-html -
### Multiple Log files ###
There are several ways to parse multiple logs with GoAccess. The simplest is to
pass multiple log files to the command line:
# goaccess access.log access.log.1
It's even possible to parse files from a pipe while reading regular files:
# cat access.log.2 | goaccess access.log access.log.1 -
**Note**: the single dash is appended to the command line to let GoAccess
know that it should read from the pipe.
Now if we want to add more flexibility to GoAccess, we can use `zcat --force`
to read compressed and uncompressed files. For instance, if we would
like to process all log files `access.log*`, we can do:
# zcat --force access.log* | goaccess -
_Note_: On Mac OS X, use `gunzip -c` instead of `zcat`.
### Multi-thread Support ###
Use `--jobs=<count>` (or `-j`) to enable multi-thread parsing. For example:
# goaccess access.log -o report.html -j 4
And use `--chunk-size=<256-32768>` to adjust chunk size, the default chunk size is 1024. For example:
# goaccess access.log -o report.html -j 4 --chunk-size=8192
### Real-time HTML outputs ###
GoAccess has the ability the output real-time data in the HTML report. You can
even email the HTML file since it is composed of a single file with no external
file dependencies, how neat is that!
The process of generating a real-time HTML report is very similar to the
process of creating a static report. Only `--real-time-html` is needed to make
it real-time.
# goaccess access.log -o /usr/share/nginx/html/your_site/report.html --real-time-html
To view the report you can navigate to `http://your_site/report.html`.
By default, GoAccess will use the host name of the generated report.
Optionally, you can specify the URL to which the client's browser will connect
to. See [FAQ](https://goaccess.io/faq) for a more detailed example.
# goaccess access.log -o report.html --real-time-html --ws-url=goaccess.io
By default, GoAccess listens on port 7890, to use a different port other than
7890, you can specify it as (make sure the port is opened):
# goaccess access.log -o report.html --real-time-html --port=9870
And to bind the WebSocket server to a different address other than 0.0.0.0, you
can specify it as:
# goaccess access.log -o report.html --real-time-html --addr=127.0.0.1
**Note**: To output real time data over a TLS/SSL connection, you need to use
`--ssl-cert=<cert.crt>` and `--ssl-key=<priv.key>`.
### Filtering ###
#### Working with dates ####
Another useful pipe would be filtering dates out of the web log
The following will get all HTTP requests starting on `05/Dec/2010` until the
end of the file.
# sed -n '/05\/Dec\/2010/,$ p' access.log | goaccess -a -
or using relative dates such as yesterdays or tomorrows day:
# sed -n '/'$(date '+%d\/%b\/%Y' -d '1 week ago')'/,$ p' access.log | goaccess -a -
If we want to parse only a certain time-frame from DATE a to DATE b, we can do:
# sed -n '/5\/Nov\/2010/,/5\/Dec\/2010/ p' access.log | goaccess -a -
If we want to preserve only certain amount of data and recycle storage, we can
keep only a certain number of days. For instance to keep & show the last 5
days:
# goaccess access.log --keep-last=5
#### Virtual hosts ####
Assuming your log contains the virtual host field. For instance:
vhost.io:80 8.8.4.4 - - [02/Mar/2016:08:14:04 -0600] "GET /shop HTTP/1.1" 200 615 "-" "Googlebot-Image/1.0"
And you would like to append the virtual host to the request in order to see
which virtual host the top urls belong to:
awk '$8=$1$8' access.log | goaccess -a -
To do the same, but also use real-time filtering and parsing:
tail -f access.log | unbuffer -p awk '$8=$1$8' | goaccess -a -
To exclude a list of virtual hosts you can do the following:
# grep -v "`cat exclude_vhost_list_file`" vhost_access.log | goaccess -
#### Files, status codes and bots ####
To parse specific pages, e.g., page views, `html`, `htm`, `php`, etc. within a
request:
# awk '$7~/\.html|\.htm|\.php/' access.log | goaccess -
Note, `$7` is the request field for the common and combined log format,
(without Virtual Host), if your log includes Virtual Host, then you probably
want to use `$8` instead. It's best to check which field you are shooting for,
e.g.:
# tail -10 access.log | awk '{print $8}'
Or to parse a specific status code, e.g., 500 (Internal Server Error):
# awk '$9~/500/' access.log | goaccess -
Or multiple status codes, e.g., all 3xx and 5xx:
# tail -f -n +0 access.log | awk '$9~/3[0-9]{2}|5[0-9]{2}/' | goaccess -o out.html -
And to get an estimated overview of how many bots (crawlers) are hitting your server:
# tail -F -n +0 access.log | grep -i --line-buffered 'bot' | goaccess -
### Tips ###
Also, it is worth pointing out that if we want to run GoAccess at lower
priority, we can run it as:
# nice -n 19 goaccess -f access.log -a
and if you don't want to install it on your server, you can still run it from
your local machine!
# ssh -n root@server 'tail -f /var/log/apache2/access.log' | goaccess - --log-format=COMBINED
**Note:** SSH requires `-n` so GoAccess can read from stdin. Also, make sure to
use SSH keys for authentication as it won't work if a passphrase is required.
#### Troubleshooting ####
We receive many questions and issues that have been answered previously.
* Date/time matching problems? Check that your log format and the system locale in which you run GoAccess match. See [#1571](https://github.com/allinurl/goaccess/issues/1571#issuecomment-543186858)
* Problems with pattern matching? Spaces are often a problem, see for instance [#136](https://github.com/allinurl/goaccess/issues/136), [#1579](https://github.com/allinurl/goaccess/issues/1579)
* Other issues matching log entries: See [>200 closed issues regarding log/date/time formats](https://github.com/allinurl/goaccess/issues?q=is%3Aissue+is%3Aclosed+label%3A%22log%2Fdate%2Ftime+format%22)
* Problems with log processing? See [>111 issues regarding log processing](https://github.com/allinurl/goaccess/issues?q=is%3Aissue+is%3Aclosed+label%3Alog-processing)
#### Incremental log processing ####
GoAccess has the ability to process logs incrementally through its internal
storage and dump its data to disk. It works in the following way:
1. A dataset must be persisted first with `--persist`, then the same dataset
can be loaded with.
2. `--restore`. If new data is passed (piped or through a log file), it will
append it to the original dataset.
##### NOTES #####
GoAccess keeps track of inodes of all the files processed (assuming files will
stay on the same partition), in addition, it extracts a snippet of data from
the log along with the last line parsed of each file and the timestamp of the
last line parsed. e.g., `inode:29627417|line:20012|ts:20171231235059`
First, it compares if the snippet matches the log being parsed, if it does, it
assumes the log hasn't changed drastically, e.g., hasn't been truncated. If
the inode does not match the current file, it parses all lines. If the current
file matches the inode, it then reads the remaining lines and updates the count
of lines parsed and the timestamp. As an extra precaution, it won't parse log
lines with a timestamp ≤ than the one stored.
Piped data works based off the timestamp of the last line read. For instance,
it will parse and discard all incoming entries until it finds a timestamp >=
than the one stored.
##### Examples #####
// last month access log
# goaccess access.log.1 --persist
then, load it with
// append this month access log, and preserve new data
# goaccess access.log --restore --persist
To read persisted data only (without parsing new data)
# goaccess --restore
## Contributing ##
Any help on GoAccess is welcome. The most helpful way is to try it out and give
feedback. Feel free to use the GitHub issue tracker and pull requests to
discuss and submit code changes.
You can contribute to our translations by editing the .po files direct on GitHub or using the visual interface [inlang.com](https://inlang.com/editor/github.com/allinurl/goaccess)
Enjoy!
SECURITY.md
+37
View File
@@ -0,0 +1,37 @@
# Security Policy
Found a security issue in [GoAccess](https://github.com/allinurl/goaccess)?
Read on.
## Reporting a Vulnerability
GoAccess takes all security bugs seriously. Thank you for improving the
security of `goaccess`. I appreciate your efforts and responsible disclosure
and will make every effort to acknowledge your contributions.
Vulnerabilities should be reported to [hello@goaccess.io](mailto:hello@goaccess.io) which is a private,
maintainer-only email address.
When I receive a security bug report, I will work on the fix and prepare a
release including the following steps:
* Confirm the problem and determine the affected versions.
* Audit the code to find any potential similar problems.
## Comments on this Policy
If you have suggestions on how this process could be improved please submit a
pull request.
When reporting an issue, where possible, please provide the following (if
possible):
* Commit version where the issue was introduced.
* A proof of concept (plaintext; or ideally send a patch to same email address)
* Steps to reproduce
* Your recommended fixes, if any.
> Note: Please do not open public issues for security issues, as GitHub does
> not provide facility for private issues, and deleting the issue makes it hard
> to triage/respond back to the reporter.
TODO
+10
View File
@@ -0,0 +1,10 @@
Copyright (C) 2009-2026
6erardo Orellana <goaccess@prosoftcorp.com>
For a more comprehensive list of to-do items, please refer to the GitHub site.
https://github.com/allinurl/goaccess/issues
or visit https://goaccess.io/faq#todo
If you are interested in working on any of the items listed in there, email
goaccess@prosoftcorp.com or better, open a new issue:
build-dynamic.sh
Executable
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env sh
# Build dynamic linked binaries on Debian.
set -o nounset ## set -u : exit the script if you try to use an uninitialised variable
set -o errexit ## set -e : exit the script if any statement returns a non-true return value
# should be either of "mmdb" or "legacy"
geoip=${1:-"mmdb"}
autoreconf -fi > /dev/null 2>&1
./configure CFLAGS=-Werror --enable-utf8 --with-openssl --with-getline --enable-geoip="$geoip" >/dev/null
make -j > /dev/null
cat goaccess
config/browsers.list
+122
View File
@@ -0,0 +1,122 @@
# List of browsers and their categories
# e.g., WORD delimited by tab(s) TYPE
# TYPE can be any type and it's not limited to the ones below.
#
# **IMPORTANT NOTE**:
# ---------------------
# The SIZE of the list is proportional to the run time.
# Thus, the longer the list, the more time GoAccess will take to parse it.
#
# Also, you should note that the higher the browser/item is on the list, the
# faster the parsing will be.
#
# The list needs to be specified using --browsers-file=<path>. This file is not
# parsed by default.
#
# The items below are sample crawlers, adjust as needed.
Chef Client Crawlers
Abonti Crawlers
SISTRIX Crawlers
DotBot Crawlers
Speedy Spider Crawlers
Sosospider Crawlers
BPImageWalker Crawlers
DoCoMo Crawlers
GSLFbot Crawlers
YodaoBot Crawlers
AddThis Crawlers
Purebot Crawlers
CCBot Crawlers
findlinks Crawlers
ichiro Crawlers
Linguee Bot Crawlers
Gigabot Crawlers
BacklinkCrawler Crawlers
distilator Crawlers
Aboundex Crawlers
UnwindFetchor Crawlers
SBIder Crawlers
TestNutch Crawlers
DomainCrawler Crawlers
NextGenSearchBot Crawlers
SEOENGWorldBot Crawlers
Cityreview Crawlers
PagePeeker Crawlers
JS-Kit Crawlers
ScreenerBot Crawlers
ShowyouBot Crawlers
SolomonoBot Crawlers
Domnutch Crawlers
MaxPoint Crawlers
NCBot Crawlers
TosCrawler Crawlers
Updownerbot Crawlers
OpenWebSpider Crawlers
WordPress Crawlers
PEAR Crawlers
ZumBot Crawlers
YisouSpider Crawlers
W3C Crawlers
vcheck Crawlers
PercolateCrawler Crawlers
NING Crawlers
gvfs Crawlers
CatchBot Crawlers
Combine Crawlers
A6-Indexer Crawlers
Altresium Crawlers
Comodo Crawlers
crawler4j Crawlers
Cricket Crawlers
EC2LinkFinder Crawlers
envolk Crawlers
GeoHasher Crawlers
HTMLParser Crawlers
MLBot Crawlers
Jaxified Crawlers
LinkWalker Crawlers
nutch Crawlers
PostRank Crawlers
keybase-proofs Crawlers
CommonCrawler Crawlers
X-CAD-SE Crawlers
Safeassign Crawlers
Nmap Crawlers
sqlmap Crawlers
Jorgee Crawlers
PxBroker Crawlers
Seekport Crawlers
adscanner Crawlers
AfD-Verbotsverfahren_JETZT! Crawlers
DuckDuckGo-favicons-Bot Crawlers
bingbot Crawlers
PetalBot Crawlers
Discordbot Crawlers
ZoominfoBot Crawlers
Googlebot Crawlers
AhrefsBot Crawlers
SemrushBot Crawlers
Adsbot Crawlers
BLEXBot Crawlers
NetcraftSurveyAgent Crawlers
Netcraft Web Server Survey Crawlers
masscan Crawlers
MJ12bot Crawlers
Pandalytics Crawlers
YandexBot Crawlers
Nimbostratus-Bot Crawlers
SeznamBot Crawlers
AppleBot Crawlers
Vienna Feeds
Windows-RSS-Platform Feeds
newsbeuter Feeds
Wrangler Feeds
Fever Feeds
Tiny Feeds
FreshRSS Feeds
KrISS Feeds
SimplePie Feeds
Feedsubs Feeds
UniversalFeedParser Feeds
config/goaccess.conf
+789
View File
@@ -0,0 +1,789 @@
######################################
# Time Format Options (required)
######################################
#
# The hour (24-hour clock) [00,23]; leading zeros are permitted but not required.
# The minute [00,59]; leading zeros are permitted but not required.
# The seconds [00,60]; leading zeros are permitted but not required.
# See `man strftime` for more details
#
# The following time format works with any of the
# Apache/NGINX's log formats below.
#
#time-format %H:%M:%S
#
# Google Cloud Storage or
# The time in microseconds since the Unix epoch.
#
#time-format %f
# Squid native log format
#
#time-format %s
######################################
# Date Format Options (required)
######################################
#
# The date-format variable followed by a space, specifies
# the log format date containing any combination of regular
# characters and special format specifiers. They all begin with a
# percentage (%) sign. See `man strftime`
#
# The following date format works with any of the
# Apache/NGINX's log formats below.
#
#date-format %d/%b/%Y
#
# AWS | Amazon CloudFront (Download Distribution)
# AWS | Elastic Load Balancing
# W3C (IIS)
#
#date-format %Y-%m-%d
#
# Google Cloud Storage or
# The time in microseconds since the Unix epoch.
#
#date-format %f
# Squid native log format
# Caddy
#
#date-format %s
######################################
# Date/Time Format Option
######################################
#
# The datetime-format variable followed by a space, specifies
# the log format date and time containing any combination of regular
# characters and special format specifiers. They all begin with a
# percentage (%) sign. See `man strftime`
#
# This gives the ability to get the timezone from a request and
# convert it to another timezone for output. See --tz=<timezone> in
# the man page.
#
#datetime-format %d/%b/%Y:%H:%M:%S %z
######################################
# Log Format Options (required)
######################################
#
# The log-format variable followed by a space or \t for
# tab-delimited, specifies the log format string.
#
# NOTE: If the time/date is a timestamp in seconds or microseconds
# %x must be used instead of %d & %t to represent the date & time.
# NCSA Combined Log Format
#log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u"
# NCSA Combined Log Format with Virtual Host
#log-format %v:%^ %h %^[%d:%t %^] "%r" %s %b "%R" "%u"
# Common Log Format (CLF)
#log-format %h %^[%d:%t %^] "%r" %s %b
# Common Log Format (CLF) with Virtual Host
#log-format %v:%^ %h %^[%d:%t %^] "%r" %s %b
# W3C
#log-format %d %t %h %^ %^ %^ %^ %r %^ %s %b %^ %^ %u %R
# Squid native log format
#log-format %^ %^ %^ %v %^: %x.%^ %~%L %h %^/%s %b %m %U
# AWS | Amazon CloudFront (Download Distribution)
#log-format %d\t%t\t%^\t%b\t%h\t%m\t%^\t%r\t%s\t%R\t%u\t%^
# Google Cloud Storage
#log-format "%x","%h",%^,%^,"%m","%U","%s",%^,"%b","%D",%^,"%R","%u"
# AWS | Elastic Load Balancing
#log-format %dT%t.%^ %^ %h:%^ %^ %T %^ %^ %^ %s %^ %b "%r" "%u"
# AWSS3 | Amazon Simple Storage Service (S3)
#log-format %^[%d:%t %^] %h %^"%r" %s %^ %b %^ %L %^ "%R" "%u"
# Virtualmin Log Format with Virtual Host
#log-format %h %^ %v %^[%d:%t %^] "%r" %s %b "%R" "%u"
# Kubernetes Nginx Ingress Log Format
#log-format %^ %^ [%h] %^ %^ [%d:%t %^] "%r" %s %b "%R" "%u" %^ %^ [%v] %^:%^ %^ %T %^ %^
# CADDY JSON Structured
#log-format {"ts":"%x.%^","request":{"client_ip":"%h","proto":"%H","method":"%m","host":"%v","uri":"%U","headers":{"User-Agent":["%u"],"Referer":["%R"]},"tls":{"cipher_suite":"%k","proto": "%K"}},"duration": "%T","size": "%b","status": "%s","resp_headers":{"Content-Type":["%M"]}}
# Traefik CLF flavor
#log-format %h - %e [%d:%t %^] "%r" %s %b "%R" "%u" %^ "%v" "%U" %Lms
# In addition to specifying the raw log/date/time formats, for
# simplicity, any of the following predefined log format names can be
# supplied to the log/date/time-format variables. GoAccess can also
# handle one predefined name in one variable and another predefined
# name in another variable.
#
#log-format COMBINED
#log-format VCOMBINED
#log-format COMMON
#log-format VCOMMON
#log-format W3C
#log-format SQUID
#log-format CLOUDFRONT
#log-format CLOUDSTORAGE
#log-format AWSELB
#log-format AWSS3
#log-format CADDY
#log-format TRAEFIKCLF
######################################
# UI Options
######################################
# Choose among color schemes
# 1 : Monochrome
# 2 : Green
# 3 : Monokai (if 256-colors supported)
#
#color-scheme 3
# Prompt log/date configuration window on program start.
#
config-dialog false
# Color highlight active panel.
#
hl-header true
# Specify a custom CSS file in the HTML report.
#
#html-custom-css /path/file.css
# Specify a custom JS file in the HTML report.
#
#html-custom-js /path/file.js
# Set default HTML preferences.
#
# NOTE: A valid JSON object is required.
# DO NOT USE A MULTILINE JSON OBJECT.
# The parser will only parse the value next to `html-prefs` (single line)
# It allows the ability to customize each panel plot. See example below.
#
#html-prefs {"theme":"bright","perPage":5,"layout":"horizontal","showTables":true,"visitors":{"plot":{"chartType":"bar"}}}
# Set HTML report page title and header.
#
#html-report-title My Awesome Web Stats
# Format JSON output using tabs and newlines.
#
json-pretty-print false
# Turn off colored output. This is the default output on
# terminals that do not support colors.
# true : for no color output
# false : use color-scheme
#
no-color false
# Don't write column names in the terminal output. By default, it displays
# column names for each available metric in every panel.
#
no-column-names false
# Disable summary metrics on the CSV output.
#
no-csv-summary false
# Disable progress metrics.
#
no-progress false
# Disable scrolling through panels on TAB.
#
no-tab-scroll false
# Disable progress metrics and parsing spinner.
#
#no-parsing-spinner true
# Do not show the last updated field displayed in the HTML generated report.
#
#no-html-last-updated true
# Outputs the report date/time data in the given timezone. Note that it
# uses the canonical timezone name. See --datetime-format in order to
# properly specify a timezone in the date/time format.
#
#tz Europe/Berlin
# Enable mouse support on main dashboard.
#
with-mouse false
# Maximum number of items to show per panel.
# Note: Only the CSV and JSON outputs allow a maximum greater than the
# default value of 366.
#
#max-items 366
# Custom colors for the terminal output
# Tailor GoAccess to suit your own tastes.
#
# Color Syntax:
# DEFINITION space/tab colorFG#:colorBG# [[attributes,] PANEL]
#
# FG# = foreground color number [-1...255] (-1 = default terminal color)
# BG# = background color number [-1...255] (-1 = default terminal color)
#
# Optionally:
#
# It is possible to apply color attributes, such as:
# bold,underline,normal,reverse,blink.
# Multiple attributes are comma separated
#
# If desired, it is possible to apply custom colors per panel, that is, a
# metric in the REQUESTS panel can be of color A, while the same metric in the
# BROWSERS panel can be of color B.
#
# The following is a 256 color scheme (hybrid palette)
#
#color COLOR_MTRC_HITS color110:color-1
#color COLOR_MTRC_VISITORS color173:color-1
#color COLOR_MTRC_DATA color221:color-1
#color COLOR_MTRC_BW color167:color-1
#color COLOR_MTRC_AVGTS color143:color-1
#color COLOR_MTRC_CUMTS color247:color-1
#color COLOR_MTRC_MAXTS color186:color-1
#color COLOR_MTRC_PROT color109:color-1
#color COLOR_MTRC_MTHD color139:color-1
#color COLOR_MTRC_HITS_PERC color186:color-1
#color COLOR_MTRC_HITS_PERC_MAX color139:color-1
#color COLOR_MTRC_HITS_PERC_MAX color139:color-1 VISITORS
#color COLOR_MTRC_HITS_PERC_MAX color139:color-1 OS
#color COLOR_MTRC_HITS_PERC_MAX color139:color-1 BROWSERS
#color COLOR_MTRC_HITS_PERC_MAX color139:color-1 VISIT_TIMES
#color COLOR_MTRC_VISITORS_PERC color186:color-1
#color COLOR_MTRC_VISITORS_PERC_MAX color139:color-1
#color COLOR_MTRC_BW_PERC color186:color-1
#color COLOR_MTRC_BW_PERC_MAX color139:color-1
#color COLOR_PANEL_COLS color243:color-1
#color COLOR_BARS color240:color-1
#color COLOR_SUBBARS color250:color-1
#color COLOR_CHART_AXIS color241:color-1
#color COLOR_CHART_VALUES color248:color-1
#color COLOR_ERROR color231:color167
#color COLOR_SELECTED color7:color167
#color COLOR_PANEL_ACTIVE color7:color237
#color COLOR_PANEL_HEADER color250:color235
#color COLOR_PANEL_DESC color242:color-1
#color COLOR_OVERALL_LBLS color243:color-1
#color COLOR_OVERALL_VALS color167:color-1
#color COLOR_OVERALL_PATH color186:color-1
#color COLOR_ACTIVE_LABEL color139:color235 bold underline
#color COLOR_BG color250:color-1
#color COLOR_DEFAULT color243:color-1
#color COLOR_PROGRESS color7:color110
######################################
# Server Options
######################################
# Specify IP address to bind server to.
#
#addr 0.0.0.0
# Run GoAccess as daemon (if --real-time-html enabled).
#
#daemonize false
# Ensure clients send the specified origin header upon the WebSocket
# handshake.
#
#origin http://example.org
# The port to which the connection is being attempted to connect.
# By default GoAccess' WebSocket server listens on port 7890
# See man page or http://gwsocket.io for details.
#
#port 7890
# Write the PID to a file when used along the daemonize option.
#
#pid-file /var/run/goaccess.pid
# Enable real-time HTML output.
#
#real-time-html true
# Path to TLS/SSL certificate.
# Note that ssl-cert and ssl-key need to be used to enable TLS/SSL.
#
#ssl-cert /path/ssl/domain.crt
# Path to TLS/SSL private key.
# Note that ssl-cert and ssl-key need to be used to enable TLS/SSL.
#
#ssl-key /path/ssl/domain.key
# URL to which the WebSocket server responds. This is the URL supplied
# to the WebSocket constructor on the client side.
#
# Optionally, it is possible to specify the WebSocket URI scheme, such as ws://
# or wss:// for unencrypted and encrypted connections.
# e.g., ws-url wss://goaccess.io
#
# If GoAccess is running behind a proxy, you could set the client side
# to connect to a different port by specifying the host followed by a
# colon and the port.
# e.g., ws-url goaccess.io:9999
#
# By default, it will attempt to connect to localhost. If GoAccess is
# running on a remote server, the host of the remote server should be
# specified here. Also, make sure it is a valid host and NOT an http
# address.
#
#ws-url goaccess.io
# Path to read named pipe (FIFO).
#
#fifo-in /tmp/wspipein.fifo
# Path to write named pipe (FIFO).
#
#fifo-out /tmp/wspipeout.fifo
## WebSocket Authentication
# See man page for more details.
#
# Enables WebSocket authentication using a JSON Web Token (JWT). Optionally, a
# secret key can be provided for verification.
#
#ws-auth=<jwt[:secret]>
# Time after which the JWT expires (only for local generated JWTs).
#
#ws-auth-expire=<secs>
# URL to fetch the initial JWT. e.g., https://site.com/api/get-auth-token
#
#ws-auth-url=<url>
# URL to fetch a new JWT when initial expires. e.g.,
# https://site.com/api/refresh-token
#
#ws-auth-refresh-url=<url>
######################################
# File Options
######################################
# Specify the path to the input log file. If set, it will take
# priority over -f from the command line.
#
#log-file /var/log/apache2/access.log
# Send all debug messages to the specified file.
#
#debug-file debug.log
# Specify a custom configuration file to use. If set, it will take
# priority over the global configuration file (if any).
#
#config-file <filename>
# Log invalid requests to the specified file.
#
#invalid-requests <filename>
# Do not load the global configuration file.
#
#no-global-config false
######################################
# Parse Options
######################################
# Enable a list of user-agents by host. For faster parsing, do not
# enable this flag.
#
agent-list false
# Enable IP resolver on HTML|JSON|CSV output.
#
with-output-resolver false
# Exclude an IPv4 or IPv6 from being counted.
# Ranges can be included as well using a dash in between
# the IPs (start-end).
#
#exclude-ip 127.0.0.1
#exclude-ip 192.168.0.1-192.168.0.100
#exclude-ip ::1
#exclude-ip 0:0:0:0:0:ffff:808:804-0:0:0:0:0:ffff:808:808
# Include HTTP request method if found. This will create a
# request key containing the request method + the actual request.
#
# <yes|no> [default: yes]
#
http-method yes
# Include HTTP request protocol if found. This will create a
# request key containing the request protocol + the actual request.
#
# <yes|no> [default: yes]
#
http-protocol yes
# Write output to stdout given one of the following files and the
# corresponding extension for the output format:
#
# /path/file.csv - Comma-separated values (CSV)
# /path/file.json - JSON (JavaScript Object Notation)
# /path/file.html - HTML
#
# output /path/file.html
# Ignore request's query string.
# i.e., www.google.com/page.htm?query => www.google.com/page.htm
#
# Note: Removing the query string can greatly decrease memory
# consumption, especially on timestamped requests.
#
no-query-string false
# Disable IP resolver on terminal output.
#
no-term-resolver false
# Treat non-standard status code 444 as 404.
#
444-as-404 false
# Add 4xx client errors to the unique visitors count.
#
4xx-to-unique-count false
# IP address anonymization
# The IP anonymization option sets the last octet of IPv4 user IP addresses and
# the last 80 bits of IPv6 addresses to zeros.
# e.g., 192.168.20.100 => 192.168.20.0
# e.g., 2a03:2880:2110:df07:face:b00c::1 => 2a03:2880:2110:df07::
#
#anonymize-ip false
# Include static files that contain a query string in the static files
# panel.
# e.g., /fonts/fontawesome-webfont.woff?v=4.0.3
#
all-static-files false
# Include an additional delimited list of browsers/crawlers/feeds etc.
# See config/browsers.list for an example or
# https://raw.githubusercontent.com/allinurl/goaccess/master/config/browsers.list
#
#browsers-file <filename>
# Date specificity. Possible values: `date` (default), or `hr` or `min`.
#
#date-spec hr|min
# Decode double-encoded values.
#
double-decode false
# Enable parsing/displaying the given panel.
#
#enable-panel VISITORS
#enable-panel REQUESTS
#enable-panel REQUESTS_STATIC
#enable-panel NOT_FOUND
#enable-panel HOSTS
#enable-panel OS
#enable-panel BROWSERS
#enable-panel VISIT_TIMES
#enable-panel VIRTUAL_HOSTS
#enable-panel REFERRERS
#enable-panel REFERRING_SITES
#enable-panel KEYPHRASES
#enable-panel STATUS_CODES
#enable-panel REMOTE_USER
#enable-panel CACHE_STATUS
#enable-panel GEO_LOCATION
#enable-panel ASN
#enable-panel MIME_TYPE
#enable-panel TLS_TYPE
# Hide a referrer but still count it. Wild cards are allowed. i.e., *.bing.com
#
#hide-referrer *.google.com
#hide-referrer bing.com
# Hour specificity. Possible values: `hr` (default), or `min` (tenth
# of a minute).
#
#hour-spec min
# Ignore crawlers from being counted.
# This will ignore robots listed under browsers.c
# Note that it will count them towards the total
# number of requests, but excluded from any of the panels.
#
ignore-crawlers false
# Parse and display crawlers only.
# This will ignore all hosts except robots listed under browsers.c
# Note that it will count them towards the total
# number of requests, but excluded from any of the panels.
#
crawlers-only false
# Unknown browsers and OS are considered as crawlers
#
unknowns-as-crawlers false
# Ignore static file requests.
# req : Only ignore request from valid requests
# panels : Ignore request from panels.
# Note that it will count them towards the total number of requests
# ignore-statics req
# Ignore parsing and displaying the given panel.
#
#ignore-panel VISITORS
#ignore-panel REQUESTS
#ignore-panel REQUESTS_STATIC
#ignore-panel NOT_FOUND
#ignore-panel HOSTS
#ignore-panel OS
#ignore-panel BROWSERS
#ignore-panel VISIT_TIMES
#ignore-panel VIRTUAL_HOSTS
ignore-panel REFERRERS
#ignore-panel REFERRING_SITES
ignore-panel KEYPHRASES
#ignore-panel STATUS_CODES
#ignore-panel REMOTE_USER
#ignore-panel CACHE_STATUS
#ignore-panel GEO_LOCATION
#ignore-panel ASN
#ignore-panel MIME_TYPE
#ignore-panel TLS_TYPE
# Ignore referrers from being counted.
# This supports wild cards. For instance,
# '*' matches 0 or more characters (including spaces)
# '?' matches exactly one character
#
#ignore-referrer *.domain.com
#ignore-referrer ww?.domain.*
# Ignore parsing and displaying one or multiple status code(s)
#
#ignore-status 400
#ignore-status 502
# Keep the last specified number of days in storage. This will recycle the
# storage tables. e.g., keep & show only the last 7 days.
#
# keep-last 7
# Disable client IP validation. Useful if IP addresses have been
# obfuscated before being logged.
#
# no-ip-validation true
# Number of lines from the access log to test against the provided
# log/date/time format. By default, the parser is set to test 10
# lines. If set to 0, the parser won't test any lines and will parse
# the whole access log.
#
#num-tests 10
# Parse log and exit without outputting data.
#
#process-and-exit false
# Display real OS names. e.g, Windows XP, Snow Leopard.
#
real-os true
# Sort panel on initial load.
# Sort options are separated by comma.
# Options are in the form: PANEL,METRIC,ORDER
#
# Available metrics:
# BY_HITS - Sort by hits
# BY_VISITORS - Sort by unique visitors
# BY_DATA - Sort by data
# BY_BW - Sort by bandwidth
# BY_AVGTS - Sort by average time served
# BY_CUMTS - Sort by cumulative time served
# BY_MAXTS - Sort by maximum time served
# BY_PROT - Sort by http protocol
# BY_MTHD - Sort by http method
# Available orders:
# ASC
# DESC
#
#sort-panel VISITORS,BY_DATA,ASC
#sort-panel REQUESTS,BY_HITS,ASC
#sort-panel REQUESTS_STATIC,BY_HITS,ASC
#sort-panel NOT_FOUND,BY_HITS,ASC
#sort-panel HOSTS,BY_HITS,ASC
#sort-panel OS,BY_HITS,ASC
#sort-panel BROWSERS,BY_HITS,ASC
#sort-panel VISIT_TIMES,BY_DATA,DESC
#sort-panel VIRTUAL_HOSTS,BY_HITS,ASC
#sort-panel REFERRERS,BY_HITS,ASC
#sort-panel REFERRING_SITES,BY_HITS,ASC
#sort-panel KEYPHRASES,BY_HITS,ASC
#sort-panel STATUS_CODES,BY_HITS,ASC
#sort-panel REMOTE_USER,BY_HITS,ASC
#sort-panel CACHE_STATUS,BY_HITS,ASC
#sort-panel GEO_LOCATION,BY_HITS,ASC
#sort-panel MIME_TYPE,BY_HITS,ASC
#sort-panel TLS_TYPE,BY_HITS,ASC
# Consider the following extensions as static files
# The actual '.' is required and extensions are case sensitive
# For a full list, uncomment the less common static extensions below.
#
static-file .css
static-file .js
static-file .jpg
static-file .png
static-file .gif
static-file .ico
static-file .jpeg
static-file .pdf
static-file .csv
static-file .mpeg
static-file .mpg
static-file .swf
static-file .woff
static-file .woff2
static-file .xls
static-file .xlsx
static-file .doc
static-file .docx
static-file .ppt
static-file .pptx
static-file .txt
static-file .zip
static-file .ogg
static-file .mp3
static-file .mp4
static-file .exe
static-file .iso
static-file .gz
static-file .rar
static-file .svg
static-file .bmp
static-file .tar
static-file .tgz
static-file .tiff
static-file .tif
static-file .ttf
static-file .flv
static-file .dmg
static-file .xz
static-file .zst
#static-file .less
#static-file .ac3
#static-file .avi
#static-file .bz2
#static-file .class
#static-file .cue
#static-file .dae
#static-file .dat
#static-file .dts
#static-file .ejs
#static-file .eot
#static-file .eps
#static-file .img
#static-file .jar
#static-file .map
#static-file .mid
#static-file .midi
#static-file .ogv
#static-file .webm
#static-file .mkv
#static-file .odp
#static-file .ods
#static-file .odt
#static-file .otf
#static-file .pict
#static-file .pls
#static-file .ps
#static-file .qt
#static-file .rm
#static-file .svgz
#static-file .wav
#static-file .webp
######################################
# GeoIP Options
# Only if configured with --enable-geoip
######################################
# To feed a database either through GeoIP Legacy or GeoIP2, you need to use the
# geoip-database flag below.
#
# === GeoIP Legacy
# Legacy GeoIP has been discontinued. If your GNU+Linux distribution does not ship
# with the legacy databases, you may still be able to find them through
# different sources. Make sure to download the .dat files.
#
# Distributed with Creative Commons Attribution-ShareAlike 4.0 International License.
# https://mailfud.org/geoip-legacy/
# IPv4 Country database:
# Download the GeoIP.dat.gz
# gunzip GeoIP.dat.gz
#
# IPv4 City database:
# Download the GeoIPCity.dat.gz
# gunzip GeoIPCity.dat.gz
# Standard GeoIP database for less memory usage (GeoIP Legacy).
#
#std-geoip false
# === GeoIP2
# For GeoIP2 databases, you can use DB-IP Lite databases.
# DB-IP is licensed under a Creative Commons Attribution 4.0 International License.
# https://db-ip.com/db/lite.php
# Or you can download them from MaxMind
# https://dev.maxmind.com/geoip/geoip2/geolite2/
# For GeoIP2 City database:
# Download the GeoLite2-City.mmdb.gz
# gunzip GeoLite2-City.mmdb.gz
#
# For GeoIP2 Country database:
# Download the GeoLite2-Country.mmdb.gz
# gunzip GeoLite2-Country.mmdb.gz
#
#geoip-database /usr/local/share/GeoIP/GeoLiteCity.dat
######################################
# Persistence Options
######################################
# Path where the persisted database files are stored on disk.
# The default value is the /tmp directory.
#db-path /tmp
# Persist parsed data into disk.
#persist true
# Load previously stored data from disk.
# Database files need to exist. See `persist`.
#restore true
config/podcast.list
+64
View File
@@ -0,0 +1,64 @@
# vim: set noexpandtab:
#1 This file is a user agent list of various podcast-related agents (but not
#2 limited to). It was distilled from the user agents of a web page which hosts
#3 a WordPress blog and a podcast.
#4 \author Bernhard R. Fischer, <bf@abenteuerland.at>
#5 \date 2019/09/26
AntennaPod Podcasts
AppleCoreMedia Mediaplayer
atc Podcasts
BacklinkCrawler Crawlers
Baiduspider Crawlers
Bullhorn Server PodCrawlers
Castbox Podcasts
CastBox Podcasts
Castro Podcasts
CCBot Crawlers
com.evolve.podcast Podcasts
cortex Facebook
ExoPlayerLib Mediaplayer
facebookexternalhit Facebook
fyyd image poller PodCrawlers
fyyd-poll PodCrawlers
icatcher Podcasts
iCatcher! Podcasts
iTMS PodCrawlers
itunesstored PodCrawlers
Kodi Mediaplayer
Lavf Mediaplayer
LCC Crawlers
Luminary Podcasts
Mediatoolkitbot Crawlers
mindUpBot Crawlers
msnbot Crawlers
NSPlayer Mediaplayer
Overcast Podcasts
Photon Crawlers
Player FM Podcasts
PlayerFM Podcasts
Plex Mediaplayer
Podbean Podcasts
podcast Podcasts
Podcast Podcasts
PodcastRepublic Podcasts
Podcasts Podcasts
Podchaser PodCrawlers
Podchaser-Parser PodCrawlers
Podnews.net PodCrawlers
PodParadise PodCrawlers
Procast Podcasts
ProCast Podcasts
Procast (iOS) Podcasts
ProcastProCast Podcasts
radio.at PodCrawlers
RadioPublic PodCrawlers
RadioPublicImageResizer PodCrawlers
RSSRadio Podcasts
Sonos Mediaplayer
Spotify PodCrawlers
stagefright Mediaplayer
Stitcher PodCrawlers
UniversalFeedParser Feeds
WinampMPEG Mediaplayer
XING FeedReader Feeds
XING-contenttabreceiver Feeds
configure.ac
+296
View File
@@ -0,0 +1,296 @@
# -*- Autoconf -*-
# Process this file with autoconf to produce a configure script.
AC_PREREQ([2.69])
AC_INIT([goaccess],[1.10.2],[hello@goaccess.io],[],[https://goaccess.io])
AM_INIT_AUTOMAKE([foreign])
AC_CONFIG_SRCDIR([src/goaccess.c])
AC_CONFIG_HEADERS([src/config.h])
# Use empty CFLAGS by default so autoconf does not add
# CFLAGS="-O2 -g"
# NOTE: Needs to go after AC_INIT and before AC_PROG_CC to select an
# empty default instead.
: ${CFLAGS=""}
# Prefer host default compiler
AC_PROG_CC([cc gcc clang])
AM_PROG_CC_C_O
# Check for programs
AM_GNU_GETTEXT([external])
AM_GNU_GETTEXT_VERSION([0.19])
# Fix `undefined reference to `libintl_gettext'` on docker:
AC_CHECK_LIB([intl], [libintl_dgettext])
# Fix undefined reference to dgettext on NetBSD
AC_CHECK_LIB([intl], [dgettext])
# pthread
AC_CHECK_LIB([pthread], [pthread_create], [], [AC_MSG_ERROR([pthread is missing])])
CFLAGS="$CFLAGS -pthread"
# DEBUG
AC_ARG_ENABLE([debug],[AS_HELP_STRING([--enable-debug],[Create a debug build. Default is disabled])],[debug="$enableval"],[debug=no])
if test "$debug" = "yes"; then
AC_DEFINE([_DEBUG], 1, [Debug option])
fi
AM_CONDITIONAL([DEBUG], [test "x$debug" = "xyes"])
# Handle rdynamic only on systems using GNU ld
AC_CANONICAL_HOST
AC_MSG_CHECKING([whether to build with rdynamic for GNU ld])
with_rdyanimc=yes
case "$host_os" in
*darwin*|*cygwin*|*aix*|*mingw*) with_rdyanimc=no
;;
esac
AC_MSG_RESULT([$with_rdyanimc])
AM_CONDITIONAL([WITH_RDYNAMIC], [test "x$with_rdyanimc" = "xyes"])
# Add ASAN
AC_CANONICAL_HOST
AC_ARG_ENABLE([asan],
[AS_HELP_STRING([--enable-asan], [Enable address sanitizer])],
[with_asan=$enableval], [with_asan=no])
AC_MSG_CHECKING([whether to build with address sanitizer])
case "$host_os" in
*cygwin*|*aix*|*mingw*) with_asan=no
;;
esac
AC_MSG_RESULT([$with_asan])
AM_CONDITIONAL([WITH_ASAN], [test "x$with_asan" = "xyes"])
# Check for libc implementation on NetBSD
AC_CHECK_HEADERS([sha.h sha1.h])
AC_CHECK_FUNCS([SHA1Init])
AM_CONDITIONAL([USE_SHA1], [test "x$ac_cv_func_SHA1Init" != "xyes"])
# Build with OpenSSL
AC_ARG_WITH([openssl],[AS_HELP_STRING([--with-openssl],[Build with OpenSSL support. Default is disabled])],[openssl="$withval"],[openssl="no"])
if test "$openssl" = 'yes'; then
AC_CHECK_LIB([ssl], [SSL_CTX_new],,[AC_MSG_ERROR([ssl library missing])])
AC_CHECK_LIB([crypto], [CRYPTO_free],,[AC_MSG_ERROR([crypto library missing])])
AC_CHECK_LIB([ssl], [SSL_CIPHER_standard_name], [AC_DEFINE([HAVE_CIPHER_STD_NAME], 1, [HAVE_CIPHER_STD_NAME])])
fi
AM_CONDITIONAL([WITH_SSL], [test "x$with_openssl" = "xyes"])
# Build with zlib
AC_ARG_WITH([zlib],
[AS_HELP_STRING([--with-zlib], [Build with zlib support for reading gzipped logs. Default is disabled])],
[zlib="$withval"],
[zlib="no"])
if test "$zlib" = 'yes'; then
AC_CHECK_LIB([z], [gzopen], [], [AC_MSG_ERROR([zlib library missing])])
AC_CHECK_HEADERS([zlib.h], [], [AC_MSG_ERROR([zlib header missing])])
AC_DEFINE([HAVE_ZLIB], 1, [Build with zlib support])
fi
AM_CONDITIONAL([WITH_ZLIB], [test "x$zlib" = "xyes"])
# GeoIP
AC_ARG_ENABLE([geoip],[AS_HELP_STRING([--enable-geoip],[Enable GeoIP country lookup. Supported types: mmdb, legacy. Default is disabled])],[geoip="$enableval"],[geoip=no])
geolocation="N/A"
if test "$geoip" = "mmdb"; then
AC_CHECK_LIB([maxminddb], [MMDB_open], [], [AC_MSG_ERROR([
*** Missing development files for libmaxminddb library.
])])
geolocation="GeoIP2"
AC_DEFINE([HAVE_GEOLOCATION], 1, [Build using GeoIP.])
elif test "$geoip" = "legacy"; then
AC_CHECK_LIB([GeoIP], [GeoIP_new], [], [AC_MSG_ERROR([
*** Missing development files for the GeoIP library
])])
geolocation="GeoIP Legacy"
AC_DEFINE([HAVE_GEOLOCATION], 1, [Build using GeoIP.])
elif test "$geoip" != "no"; then
AC_MSG_ERROR([*** Invalid argument for GeoIP: $geoip])
fi
AM_CONDITIONAL([GEOIP_LEGACY], [test "x$geoip" = "xlegacy"])
AM_CONDITIONAL([GEOIP_MMDB], [test "x$geoip" = "xmmdb"])
# GNU getline / POSIX.1-2008
AC_ARG_WITH([getline],[AS_HELP_STRING([--with-getline],[Build using dynamic line buffer. Default is disabled])],[with_getline=$withval],[with_getline=no])
if test "$with_getline" = "yes"; then
AC_DEFINE([WITH_GETLINE], 1, [Build using GNU getline.])
fi
# UTF8
AC_ARG_ENABLE([utf8],[AS_HELP_STRING([--enable-utf8],[Enable ncurses library that handles wide characters. Default is disabled])],[utf8="$enableval"],[utf8=no])
if test "$utf8" = "yes"; then
libncursesw=ncursesw
# Simply called libncurses on OS X
case "$host_os" in
*darwin*) libncursesw=ncurses
;;
esac
AC_CHECK_LIB([$libncursesw], [mvaddwstr], [],
[AC_MSG_ERROR([*** Missing development libraries for ncursesw])])
AC_SEARCH_LIBS([tputs], [tinfow], ,[AC_MSG_ERROR([Cannot find a library providing tputs])])
AC_DEFINE([HAVE_LIBNCURSESW], [1], ["ncursesw is present."])
have_ncurses="yes"
AC_CHECK_HEADERS([ncursesw/ncurses.h],[have_ncurses=yes], [], [
#ifdef HAVE_NCURSESW_NCURSES_H
#include <ncursesw/ncurses.h>
#endif
])
AC_CHECK_HEADERS([ncurses.h],[have_ncurses=yes], [], [
#ifdef HAVE_NCURSES_H
#include <ncurses.h>
#endif
])
if test "$have_ncurses" != "yes"; then
AC_MSG_ERROR([Missing ncursesw header file])
fi
else
AC_CHECK_LIB([ncurses], [refresh], [],
[AC_CHECK_LIB([curses], [refresh], [],
[AC_MSG_ERROR([*** Missing development libraries for ncurses])])])
AC_SEARCH_LIBS([tputs], [tinfo], ,[AC_MSG_ERROR([Cannot find a library providing tputs])])
have_ncurses="yes"
AC_CHECK_HEADERS([ncurses/ncurses.h],[have_ncurses=yes], [], [
#ifdef HAVE_NCURSES_NCURSES_H
#include <ncurses/ncurses.h>
#endif
])
AC_CHECK_HEADERS([ncurses.h],[have_ncurses=yes], [], [
#ifdef HAVE_NCURSES_H
#include <ncurses.h>
#endif
])
AC_CHECK_HEADERS([curses.h],[have_ncurses=yes], [], [
#ifdef HAVE_CURSES_H
#include <curses.h>
#endif
])
if test "$have_ncurses" != "yes"; then
AC_MSG_ERROR([Missing ncurses header file])
fi
fi
# Default Hash
storage="In-Memory with On-Disk Persistent Storage"
HAS_SEDTR=no
AC_CHECK_PROG([SED_CHECK],[sed],[yes],[no])
if test x"$SED_CHECK" = x"yes" ; then
AC_CHECK_PROG([TR_CHECK],[tr],[yes],[no])
if test x"$TR_CHECK" = x"yes" ; then
HAS_SEDTR=yes
fi
fi
AM_CONDITIONAL([HAS_SEDTR], [test "x$HAS_SEDTR" = xyes])
# detect Cygwin or MinGW and use mmap family replacements
USE_MMAP=no
case $host in
*-*-mingw32* | *-*-cygwin* | *-*-windows*)
USE_MMAP=yes
AC_MSG_NOTICE([using custom mmap for Cygwin/MinGW])
;;
esac
AM_CONDITIONAL([USE_MMAP], [test "x$USE_MMAP" = xyes])
# Solaris
AC_CHECK_LIB([nsl], [gethostbyname])
AC_CHECK_LIB([socket], [socket])
# Checks for header files.
AC_HEADER_STDC
AC_CHECK_HEADERS([arpa/inet.h])
AC_CHECK_HEADERS([fcntl.h])
AC_CHECK_HEADERS([inttypes.h])
AC_CHECK_HEADERS([limits.h])
AC_CHECK_HEADERS([locale.h])
AC_CHECK_HEADERS([netdb.h])
AC_CHECK_HEADERS([netinet/in.h])
AC_CHECK_HEADERS([stddef.h])
AC_CHECK_HEADERS([stdint.h])
AC_CHECK_HEADERS([stdlib.h])
AC_CHECK_HEADERS([string.h])
AC_CHECK_HEADERS([strings.h])
AC_CHECK_HEADERS([sys/socket.h])
AC_CHECK_HEADERS([sys/time.h])
AC_CHECK_HEADERS([unistd.h])
# Checks for typedefs, structures, and compiler characteristics.
AC_C_CONST
AC_CHECK_TYPES([ptrdiff_t])
AC_STRUCT_TM
AC_TYPE_INT64_T
AC_TYPE_INT8_T
AC_TYPE_OFF_T
AC_TYPE_SIZE_T
AC_TYPE_UINT32_T
AC_TYPE_UINT64_T
AC_TYPE_UINT8_T
# Checks for library functions.
AC_FUNC_FSEEKO
AC_FUNC_MEMCMP
AC_FUNC_MKTIME
AC_FUNC_STAT
AC_FUNC_STRFTIME
AC_FUNC_STRTOD
AC_CHECK_FUNCS([floor])
AC_CHECK_FUNCS([gethostbyaddr])
AC_CHECK_FUNCS([gethostbyname])
AC_CHECK_FUNCS([gettimeofday])
AC_CHECK_FUNCS([malloc])
AC_CHECK_FUNCS([memmove])
AC_CHECK_FUNCS([memset])
AC_CHECK_FUNCS([mkfifo])
AC_CHECK_FUNCS([poll])
AC_CHECK_FUNCS([realloc])
AC_CHECK_FUNCS([realpath])
AC_CHECK_FUNCS([regcomp])
AC_CHECK_FUNCS([setlocale])
AC_CHECK_FUNCS([socket])
AC_CHECK_FUNCS([strcasecmp])
AC_CHECK_FUNCS([strchr])
AC_CHECK_FUNCS([strcspn])
AC_CHECK_FUNCS([strdup])
AC_CHECK_FUNCS([strerror])
AC_CHECK_FUNCS([strncasecmp])
AC_CHECK_FUNCS([strpbrk])
AC_CHECK_FUNCS([strrchr])
AC_CHECK_FUNCS([strspn])
AC_CHECK_FUNCS([strstr])
AC_CHECK_FUNCS([strtol])
AC_CHECK_FUNCS([strtoull])
AC_CHECK_FUNCS([timegm])
AC_CONFIG_FILES([Makefile po/Makefile.in])
AC_OUTPUT
cat << EOF
Your build configuration:
Prefix : $prefix
Package : $PACKAGE_NAME
Version : $VERSION
Compiler flags : $CFLAGS
Linker flags : $LIBS $LDFLAGS
UTF-8 support : $utf8
Dynamic buffer : $with_getline
ASan : $with_asan
Geolocation : $geolocation
Storage method : $storage
TLS/SSL : $openssl
zlib support : $zlib
Bugs : $PACKAGE_BUGREPORT
EOF
docker-compose/README.md
+34
View File
@@ -0,0 +1,34 @@
# Docker-compose configuration
Here are two docker-compose configurations for goaccess which
combine a static site with a real-time report of goaccess.
The directories used are:
- `configs` - for the nginx and goaccess configuration
- `public` - the files served by nginx - put your static site here
- `logs` - nginx logs - there is no log rotation in place!
There are two flavors of the docker-compose files and the goaccess files:
- `*.vanilla.*` which need you to take care of the TLS certificates
- `*.traefik.*` using traefik for TLS and domain routing
## Vanilla
For the vanilla version, you'll have to do the following:
- put the TLS certificates into `configs/certs`, following the naming scheme
in `goaccess.vanilla.conf`
- route requests for the static webpage to port `8080`
## Traefik
The traefik version is setup to make it easier to do the routing.
You don't need to take care of the TLS certificates, and the
goaccess websocket gets its own subdomain.
To put it all together, the following environment is needed:
- traefik configured according to [Traefik-101](https://ineiti.ch/posts/traefik-101/traefik-101/)
- DNS configuration for two domain names pointing to your server's IP:
- `yourdomain` for the static pages, e.g., a blog-post using [Hugo](https://gohugo.io/)
- `goaccess.yourdomain` for the stats with goaccess
docker-compose/configs/goaccess.traefik.conf
+9
View File
@@ -0,0 +1,9 @@
tz Europe/Zurich
time-format %H:%M:%S
date-format %Y-%m-%d
log-format COMBINED
log-file /srv/logs/access.log
output /srv/report/report.html
real-time-html true
ws-url wss://goaccess.yourdomain
port 443
docker-compose/configs/goaccess.vanilla.conf
+10
View File
@@ -0,0 +1,10 @@
tz Europe/Zurich
time-format %H:%M:%S
date-format %Y-%m-%d
log-format COMBINED
log-file /srv/logs/access.log
output /srv/report/report.html
real-time-html true
ws-url wss://yourdomain
ssl-cert /srv/certs/yourdomain.crt
ssl-key /srv/certs/yourdomain.key
docker-compose/configs/nginx.conf
+31
View File
@@ -0,0 +1,31 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
docker-compose/docker-compose.traefik.yaml
+33
View File
@@ -0,0 +1,33 @@
services:
web:
image: nginx
volumes:
- ./logs:/var/log/nginx
- ./configs/nginx.conf:/etc/nginx/nginx.conf
- ./public:/usr/share/nginx/html
labels:
- "traefik.enable=true"
- "fqdn=yourdomain"
networks:
- traefik
goaccess:
image: allinurl/goaccess
container_name: goaccess
volumes:
- ./configs/goaccess.traefik.conf:/srv/config/goaccess.conf
- ./logs:/srv/logs
- ./public:/srv/report
command: ["--no-global-config", "--config-file=/srv/config/goaccess.conf"]
labels:
- "traefik.enable=true"
- "fqdn=goaccess.yourdomain"
- "traefik.http.services.yourdomain-goaccess.loadbalancer.server.port=443"
restart: unless-stopped
networks:
- traefik
networks:
traefik:
external:
name: traefik_traefik
docker-compose/docker-compose.vanilla.yaml
+21
View File
@@ -0,0 +1,21 @@
services:
web:
image: nginx
ports:
- 8080:80
volumes:
- ./logs:/var/log/nginx
- ./configs/nginx.conf:/etc/nginx/nginx.conf
- ./public:/usr/share/nginx/html
goaccess:
image: allinurl/goaccess
ports:
- 7890:7890
volumes:
- ./configs/goaccess.vanilla.conf:/srv/config/goaccess.conf
- ./logs:/srv/logs
- ./public:/srv/report
- ./configs/certs:/srv/certs
command: ["--no-global-config", "--config-file=/srv/config/goaccess.conf"]
restart: unless-stopped
goaccess.1
+1540
View File
File diff suppressed because it is too large Load Diff
inlang.config.js
+46
View File
@@ -0,0 +1,46 @@
// init the inlang.config
/**
* @type {import("@inlang/core/config").DefineConfig}
*/
export async function defineConfig(env) {
// importing plugin from local file for testing purposes
const plugin = await env.$import(
"https://cdn.jsdelivr.net/gh/jannesblobel/inlang-plugin-po@1/dist/index.js"
);
const pluginConfig = {
// language mean the name of you file
pathPattern: "./po/{language}.po",
referenceResourcePath: "./po/goaccess.pot",
};
return {
// if your project use a pot file use the pot as the reference Language
// !! do not add the pot file in the Languages array
/**
* @example
* example files: en.pot, de.po, es.po, fr.po
* referenceLanguage: "en",
languages: ["de","es","fr"],
*/
referenceLanguage: "en",
languages: await getLanguages(env),
readResources: (args) =>
plugin.readResources({ ...args, ...env, pluginConfig }),
writeResources: (args) =>
plugin.writeResources({ ...args, ...env, pluginConfig }),
};
}
/**
* Automatically derives the languages in this repository.
*/
async function getLanguages(env) {
const files = await env.$fs.readdir("./po");
// files that end with .json
// remove the .json extension to only get language name
const languages = files
.filter((name) => name.endsWith(".po"))
.map((name) => name.replace(".po", ""));
return languages;
}
po/LINGUAS
+13
View File
@@ -0,0 +1,13 @@
de
es
fr
ja
ka
ko
sv
zh_CN
pt_BR
ru
uk
it
po/Makevars
+79
View File
@@ -0,0 +1,79 @@
# Makefile variables for PO directory in any package using GNU gettext.
# Usually the message domain is the same as the package name.
DOMAIN = $(PACKAGE)
# These two variables depend on the location of this directory.
subdir = po
top_builddir = ..
AM_CPPFLAGS = -I. -I$(srcdir)
# These options get passed to xgettext.
XGETTEXT_OPTIONS = --from-code=UTF-8 --keyword=_ --keyword=N_
# This is the copyright holder that gets inserted into the header of the
# $(DOMAIN).pot file. Set this to the copyright holder of the surrounding
# package. (Note that the msgstr strings, extracted from the package's
# sources, belong to the copyright holder of the package.) Translators are
# expected to transfer the copyright for their translations to this person
# or entity, or to disclaim their copyright. The empty string stands for
# the public domain; in this case the translators are expected to disclaim
# their copyright.
COPYRIGHT_HOLDER = Free Software Foundation, Inc.
# This tells whether or not to prepend "GNU " prefix to the package
# name that gets inserted into the header of the $(DOMAIN).pot file.
# Possible values are "yes", "no", or empty. If it is empty, try to
# detect it automatically by scanning the files in $(top_srcdir) for
# "GNU packagename" string.
PACKAGE_GNU =
# This is the email address or URL to which the translators shall report
# bugs in the untranslated strings:
# - Strings which are not entire sentences, see the maintainer guidelines
# in the GNU gettext documentation, section 'Preparing Strings'.
# - Strings which use unclear terms or require additional context to be
# understood.
# - Strings which make invalid assumptions about notation of date, time or
# money.
# - Pluralisation problems.
# - Incorrect English spelling.
# - Incorrect formatting.
# It can be your email address, or a mailing list address where translators
# can write to without being subscribed, or the URL of a web page through
# which the translators can contact you.
MSGID_BUGS_ADDRESS = hello@goaccess.io
# This is the list of locale categories, beyond LC_MESSAGES, for which the
# message catalogs shall be used. It is usually empty.
EXTRA_LOCALE_CATEGORIES =
# This tells whether the $(DOMAIN).pot file contains messages with an 'msgctxt'
# context. Possible values are "yes" and "no". Set this to yes if the
# package uses functions taking also a message context, like pgettext(), or
# if in $(XGETTEXT_OPTIONS) you define keywords with a context argument.
USE_MSGCTXT = no
# These options get passed to msgmerge.
# Useful options are in particular:
# --previous to keep previous msgids of translated messages,
# --quiet to reduce the verbosity.
MSGMERGE_OPTIONS =
# These options get passed to msginit.
# If you want to disable line wrapping when writing PO files, add
# --no-wrap to MSGMERGE_OPTIONS, XGETTEXT_OPTIONS, and
# MSGINIT_OPTIONS.
MSGINIT_OPTIONS =
# This tells whether or not to regenerate a PO file when $(DOMAIN).pot
# has changed. Possible values are "yes" and "no". Set this to no if
# the POT file is checked in the repository and the version control
# program ignores timestamps.
PO_DEPENDS_ON_POT = yes
# This tells whether or not to forcibly update $(DOMAIN).pot and
# regenerate PO files on "make dist". Possible values are "yes" and
# "no". Set this to no if the POT file and PO files are maintained
# externally.
DIST_DEPENDS_ON_UPDATE_PO = yes
po/POTFILES.in
+2
View File
@@ -0,0 +1,2 @@
# List of source files which contain translatable strings.
src/labels.h
po/de.po
+1131
View File
File diff suppressed because it is too large Load Diff
po/es.po
+1096
View File
File diff suppressed because it is too large Load Diff
po/fr.po
+1108
View File
File diff suppressed because it is too large Load Diff
po/goaccess.pot
+1080
View File
File diff suppressed because it is too large Load Diff
po/it.po
+1104
View File
File diff suppressed because it is too large Load Diff
po/ja.po
+1123
View File
File diff suppressed because it is too large Load Diff
po/ka.po
+1119
View File
File diff suppressed because it is too large Load Diff
po/ko.po
+1106
View File
File diff suppressed because it is too large Load Diff
po/pt_BR.po
+1111
View File
File diff suppressed because it is too large Load Diff
po/ru.po
+1114
View File
File diff suppressed because it is too large Load Diff
po/sv.po
+1120
View File
File diff suppressed because it is too large Load Diff
po/uk.po
+1114
View File
File diff suppressed because it is too large Load Diff
po/zh_CN.po
+1078
View File
File diff suppressed because it is too large Load Diff
resources/cities-10m.json
+1
View File
File diff suppressed because one or more lines are too long
resources/countries-110m.json
+1
View File
File diff suppressed because one or more lines are too long
resources/css/app.css
+1759
View File
File diff suppressed because it is too large Load Diff
resources/css/bootstrap.min.css
Vendored
+6
View File
File diff suppressed because one or more lines are too long
resources/css/fa.min.css
Vendored
+407
View File
File diff suppressed because one or more lines are too long
resources/goaccess.svg
+12
View File
@@ -0,0 +1,12 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32">
<path fill="#1b1b1b" d="M0 16V0h32v32H0Z"/>
<path fill="#afd42a" d="M8.075 20.932V19.92h4.05v2.025h-4.05Zm0 4.05V23.97h4.05v2.026h-4.05Zm0 4.05v-1.013h4.05v2.025h-4.05Z"/>
<path fill="#fb1161" d="M2 20.932V19.92h4.05v2.025H2Zm0 4.05V23.97h4.05v2.026H2Zm0 4.05v-1.013h4.05v2.025H2Z"/>
<path fill="#585858" d="M0 16.2v-2.025h32v4.05H0ZM0 2.024V0h32v4.05H0Z"/>
<path fill="#65d7fd" d="M14.27 29.057v-.943h3.905V30H14.27zm0-8.08v-.942h3.905v1.885H14.27zm0 4.04v-.942h3.905v1.885H14.27z"/>
<path fill="#c6c6c6" d="M20.33 29.057v-.943h3.904V30H20.33zm0-4.04v-.942h7.944v1.885H20.33zm0-4.04v-.942h5.924v1.885H20.33z"/>
<path fill="#afd42a" d="M10.1 7.013V6h8.1v2.025h-8.1z"/>
<path fill="#c6c6c6" d="M2.016 7.016v-1.01h6.06v2.02h-6.06z"/>
<path fill="#afd42a" d="M10.1 11.063V10.05h8.1v2.025h-8.1z"/>
<path fill="#c6c6c6" d="M2.016 11.066v-1.01h6.06v2.02h-6.06z"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 946 B

resources/js/app.js
+2445
View File
File diff suppressed because it is too large Load Diff
resources/js/charts.js
+1857
View File
File diff suppressed because it is too large Load Diff
resources/js/d3.v7.min.js
Vendored
+2
View File
File diff suppressed because one or more lines are too long
resources/js/hogan.min.js
Vendored
+5
View File
File diff suppressed because one or more lines are too long
resources/js/topojson.v3.min.js
Vendored
+2
View File
File diff suppressed because one or more lines are too long
resources/tpls.html
+336
View File
@@ -0,0 +1,336 @@
<!-- TPL General -->
<script id="tpl-general" type="text/template">
<hgroup>
<h2 class="gheader h4" id="overall-heading">{{head}}</h2>
<p>{{#from}}{{from}}{{/from}}{{#to}} – {{to}}{{/to}}</p>
</hgroup>
<ul role="list" class="row"></ul>
</script>
<!-- TPL General Items -->
<script id="tpl-general-items" type="text/template">
<li class="col-sm-6 col-md-2 {{#className}}{{className}}{{/className}}{{^className}}gray{{/className}}" aria-live="polite" aria-atomic="true">
<div>
<span><i class="fa fa-bar-chart"></i> {{#label}}{{label}}{{/label}}</span>
<span id="{{id}}">{{value}}</span>
</div>
</li>
</script>
<!-- TPL Panel Table -->
<script id="tpl-table-row" type="text/template">
{{#rows}}
<tr class="{{#className}}{{className}} {{/className}}{{#hasSubItems}}{{#items}}expandable{{/items}}{{/hasSubItems}} level-{{level}}" data-panel="{{panel}}" {{#key}}data-key="{{key}}" {{/key}} {{#nodeKey}}data-node-key="{{nodeKey}}" {{/nodeKey}} data-level="{{level}}">
{{#hasSubItems}}
<td class="row-expandable text-center {{#items}}clickable{{/items}}">
{{#items}}<i class="fa {{#expanded}}fa-caret-down{{/expanded}}{{^expanded}}fa fa-caret-right{{/expanded}}"></i>{{/items}}
{{^items}}<i></i>{{/items}}
</td>
{{/hasSubItems}}
{{^hasSubItems}}
{{#showPlaceholder}}
<td class="row-expandable text-center">
<i></i>
</td>
{{/showPlaceholder}}
{{/hasSubItems}}
<td class="row-idx text-right">
{{#idx}}{{idx}}{{/idx}}
</td>
{{#cells}}
<td class="{{className}}" {{#colspan}}colspan="{{colspan}}"{{/colspan}}>
<span class="value">{{{value}}}</span>{{#percent}}<span class="percent"> ({{percent}})</span>{{/percent}}
</td>
{{/cells}}
</tr>
{{/rows}}
</script>
<!-- TPL Panel Table Meta -->
<script id="tpl-table-row-meta" type="text/template">
{{#row}}
{{#hasSubItems}}
<th class=""></th>
{{/hasSubItems}}
<th class="meta-label">{{key}}.</th>
{{#cells}}
<th class="{{className}}" {{#colspan}}colspan="{{colspan}}"{{/colspan}}>
{{#value}}
<span>{{value}} {{#percent}}<span class="text-muted">({{percent}}%)</span>{{/percent}}</span>
{{/value}}
{{^value}}
<span class="text-muted">&mdash;</span>
{{/value}}
</th>
{{/cells}}
{{/row}}
</script>
<!-- TPL Panel Table Totals -->
<script id="tpl-table-row-totals" type="text/template">
{{#row}}
{{#hasSubItems}}
<td class=""></td>
{{/hasSubItems}}
<td class="meta-label">Tot.</td>
{{#cells}}
<td class="{{className}}" {{#colspan}}colspan="{{colspan}}"{{/colspan}}>
{{#label}}
<span>{{label}}</span>&nbsp;
{{/label}}
<span>{{value}}</span>
</td>
{{/cells}}
{{/row}}
</script>
<!-- TPL Table thead -->
<script id="tpl-table-thead" type="text/template">
{{#hasSubItems}}
<th></th>
{{/hasSubItems}}
<th>#</th>
{{#items}}
<th class="{{dataType}} {{#key}}sortable{{/key}}" data-key="{{key}}" {{#sort}}data-order="{{#asc}}asc{{/asc}}{{^asc}}desc{{/asc}}"{{/sort}}>
{{label}} <i class="fa fa-{{^sort}}sort{{/sort}}{{#sort}}{{#asc}}caret-up{{/asc}}{{^asc}}caret-down{{/asc}}{{/sort}}"></i>
</th>
{{/items}}
</script>
<!-- TPL Panel Options DropDown -->
<script id="tpl-panel-opts" type="text/template">
<li class="dropdown-header">{{ labels.chart_opts }}</li>
<li><a href="javascript:void(0);" data-panel="{{id}}" data-chart="{{showChart}}"><i class="fa fa-{{#showChart}}check-{{/showChart}}square-o"></i> {{labels.chart}}</a></li>
{{#plot.length}}
<li class="dropdown-header">{{ labels.type }}</li>
<li><a href="javascript:void(0);" data-panel="{{id}}" data-chart-type="area-spline"><i class="fa fa-circle{{^area-spline}}-o{{/area-spline}}"></i> {{labels.area_spline}}</a></li>
<li><a href="javascript:void(0);" data-panel="{{id}}" data-chart-type="bar"><i class="fa fa-circle{{^bar}}-o{{/bar}}"></i> {{labels.bar}}</a></li>
{{#hasMap}}
<li><a href="javascript:void(0);" data-panel="{{id}}" data-chart-type="wmap"><i class="fa fa-circle{{^wmap}}-o{{/wmap}}"></i> {{labels.wmap}}</a></li>
<li><a href="javascript:void(0);" data-panel="{{id}}" data-chart-type="gmap"><i class="fa fa-circle{{^gmap}}-o{{/gmap}}"></i> {{labels.gmap}}</a></li>
{{/hasMap}}
<li class="dropdown-header">{{labels.plot_metric}}</li>
{{#plot}}
<li><a href="javascript:void(0);" data-panel="{{id}}" data-plot="{{className}}" class="panel-plot-{{className}}"><i class="fa fa-circle{{^selected}}-o{{/selected}}"></i> {{label}}</a></li>
{{/plot}}
{{/plot.length}}
<li class="dropdown-header">{{labels.table_columns}}</li>
{{#items}}
<li><a href="javascript:void(0);" data-panel="{{id}}" data-metric="{{key}}"><i class="fa fa-{{^hide}}check-{{/hide}}square-o"></i> {{label}}</a></li>
{{/items}}
</script>
<!-- TPL Table colgroup -->
<script id="tpl-table-colgroup" type="text/template">
{{#hasSubItems}}
<col style="width: 2%;"> <!-- right-caret -->
{{/hasSubItems}}
<col style="width: 3%;"> <!-- row # -->
{{#items}}
<col style="width:{{colWidth}}">
{{/items}}
</script>
<!-- TPL Panel -->
<script id="tpl-panel" type="text/template">
<div class="row">
<div class="col-md-12">
<header>
<hgroup>
<h2 id="{{id}}" class="gheader">{{head}}</h2>
<small class="text-muted">{{desc}}</small>
</hgroup>
<div>
<div class="dropdown">
<button class="btn btn-default btn-sm dropdown-toggle" type="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false" data-panel="{{id}}" aria-label="{{labels.panel_opts}}, {{head}}">
<i class="fa fa-gear" aria-hidden="true"></i> {{labels.panel_opts}} <span class="fa fa-caret-down" aria-hidden="true"></span>
</button>
<ul class="dropdown-menu dropdown-menu-right panel-opts-{{id}}">
</ul>
</div>
</div>
</header>
</div>
</div>
{{#chart}}
<div class="row">
<div class="col-md-12">
<div id="chart-{{id}}" class="chart-wrap"></div>
</div>
</div>
{{/chart}}
{{#table}}
<div class="row clearfix table-wrapper {{#autoHideTables}}hidden-xs{{/autoHideTables}}">
<div class="col-md-12">
<div class="table-responsive">
<table data-panel="{{id}}" class="table table-borderless table-hover table-{{id}}">
<colgroup>
</colgroup>
<thead >
<tr class="thead-cols"></tr>
<tr class="thead-min"></tr>
<tr class="thead-max"></tr>
<tr class="thead-avg"></tr>
</thead>
<tbody class="tbody-data">
</tbody>
<tfoot class="tfoot-totals">
<tr class="tfoot-totals">
</tr>
</tfoot>
</table>
</div>
<ul class="pagination pagination-sm pull-left">
<li class="disabled">
<a class="panel-first" href="javascript:void(0);" aria-label="{{labels.first}}" data-panel="{{id}}" title="{{labels.first}}">
<i class="fa fa-angle-double-left"></i>
</a>
</li>
<li class="disabled">
<a class="panel-prev" href="javascript:void(0);" aria-label="{{labels.previous}}" data-panel="{{id}}" title="{{labels.previous}}">
<i class="fa fa-angle-left"></i>
</a>
</li>
<li>
<a class="panel-next" href="javascript:void(0);" aria-label="{{labels.next}}" data-panel="{{id}}" title="{{labels.next}}">
<i class="fa fa-angle-right"></i>
</a>
</li>
<li>
<a class="panel-last" href="javascript:void(0);" aria-label="{{labels.last}}" data-panel="{{id}}" title="{{labels.last}}">
<i class="fa fa-angle-double-right"></i>
</a>
</li>
</ul>
</div>
</div>
{{/table}}
</script>
<!-- TPL Nav Bar wrapper -->
<script id="tpl-nav-wrap" type="text/template">
<button type="button"class="nav-bars" aria-label="{{ button_menu }}"><i class="fa fa-bars" aria-hidden="true"></i></button>
<button type="button"class="nav-gears" aria-label="{{ button_settings }}"><i class="fa fa-cog" aria-hidden="true"></i></button>
<i class="nav-ws-status fa fa-stop" aria-live="polite" aria-label="{{ websocket_disconnected }}" title="{{ websocket_disconnected }}"></i>
<div class="nav-list"></div>
<div class="powered hidden-xs hidden-sm">by <a href="https://goaccess.io/">GoAccess</a> <span>v{{version}}</span> and <a href="https://gwsocket.io/">GWSocket</a></div>
</script>
<!-- TPL Nav Bar items -->
<script id="tpl-nav-menu" type="text/template">
<h3>{{labels.panels}}</h3>
<ul>
<!-- Overall stats - not draggable -->
<li {{#overall_current}}class="active"{{/overall_current}}>
<span class="drag-handle" style="visibility: hidden;">
<i class="fa fa-bars"></i>
</span>
<a href="#"><i class="fa fa-bar-chart"></i> {{labels.thead}}</a>
<span title="{{labels.toggle_panel}}" class="toggle-panel {{#overall_hidden}}active{{/overall_hidden}}" data-panel="general"><i class="fa fa-circle-o"></i></span>
</li>
<!-- Panel items - draggable -->
{{#nav}}
<li {{#current}}class="active"{{/current}}>
<span class="drag-handle" title="{{labels.drag_to_reorder}}">
<i class="fa fa-bars"></i>
</span>
<a href="#{{key}}"><i class="fa fa-{{icon}}"></i> {{head}}</a>
<span title="{{labels.toggle_panel}}" class="toggle-panel {{#hidden}}active{{/hidden}}" data-panel="{{key}}"><i class="fa fa-circle-o"></i></span>
</li>
{{/nav}}
</ul>
</script>
<!-- TPL Nav Bar options -->
<script id="tpl-nav-opts" type="text/template">
<h3><i class="fa fa-hashtag"></i> {{labels.theme}}</h3>
<ul>
<li {{#darkGray}}class="active"{{/darkGray}}>
<a href="javascript:void(0);" class="theme-dark-gray"><i class="fa fa-circle{{^darkGray}}-o{{/darkGray}}"></i> {{labels.dark_gray}}</a>
</li>
<li {{#bright}}class="active"{{/bright}}>
<a href="javascript:void(0);" class="theme-bright"><i class="fa fa-circle{{^bright}}-o{{/bright}}"></i> {{labels.bright}}</a>
</li>
<li {{#darkBlue}}class="active"{{/darkBlue}}>
<a href="javascript:void(0);" class="theme-dark-blue"><i class="fa fa-circle{{^darkBlue}}-o{{/darkBlue}}"></i> {{labels.dark_blue}}</a>
</li>
<li {{#darkPurple}}class="active"{{/darkPurple}}>
<a href="javascript:void(0);" class="theme-dark-purple"><i class="fa fa-circle{{^darkPurple}}-o{{/darkPurple}}"></i> {{labels.dark_purple}}</a>
</li>
</ul>
<h3><i class="fa fa-list-alt"></i> {{labels.panels}}</h3>
<ul class="perpage-wrap">
<li class="dropdown-header"><i class="fa fa-list"></i> {{labels.items_per_page}}</li>
<li {{#perPage3}}class="active"{{/perPage3}}>
<a href="javascript:void(0);" data-perpage="3"><i class="fa fa-circle{{^perPage3}}-o{{/perPage3}}"></i> 3</a>
</li>
<li {{#perPage5}}class="active"{{/perPage5}}>
<a href="javascript:void(0);" data-perpage="5"><i class="fa fa-circle{{^perPage5}}-o{{/perPage5}}"></i> 5</a>
</li>
<li {{#perPage7}}class="active"{{/perPage7}}>
<a href="javascript:void(0);" data-perpage="7"><i class="fa fa-circle{{^perPage7}}-o{{/perPage7}}"></i> 7</a>
</li>
<li {{#perPage10}}class="active"{{/perPage10}}>
<a href="javascript:void(0);" data-perpage="10"><i class="fa fa-circle{{^perPage10}}-o{{/perPage10}}"></i> 10</a>
</li>
<li {{#perPage15}}class="active"{{/perPage15}}>
<a href="javascript:void(0);" data-perpage="15"><i class="fa fa-circle{{^perPage15}}-o{{/perPage15}}"></i> 15</a>
</li>
<li {{#perPage20}}class="active"{{/perPage20}}>
<a href="javascript:void(0);" data-perpage="20"><i class="fa fa-circle{{^perPage20}}-o{{/perPage20}}"></i> 20</a>
</li>
<li {{#perPage24}}class="active"{{/perPage24}}>
<a href="javascript:void(0);" data-perpage="24"><i class="fa fa-circle{{^perPage24}}-o{{/perPage24}}"></i> 24</a>
</li>
<li class="dropdown-header"><i class="fa fa-table"></i> {{labels.tables}}</li>
<li {{#showTables}}class="active"{{/showTables}}>
<a href="javascript:void(0);" data-show-tables="1"><i class="fa fa-{{#showTables}}check-{{/showTables}}square-o"></i> {{labels.display_tables}}</a>
</li>
<li {{#autoHideTables}}class="active"{{/autoHideTables}}>
<a href="javascript:void(0);" data-autohide-tables="1" title="{{labels.ah_small_title}}">
<i class="fa fa-{{#autoHideTables}}check-{{/autoHideTables}}square-o"></i> {{labels.ah_small}}
</a>
</li>
</ul>
<h3><i class="fa fa-th-large"></i> {{labels.layout}}</h3>
<ul>
<li {{#horizontal}}class="active"{{/horizontal}}>
<a href="javascript:void(0);" class="layout-horizontal"><i class="fa fa-circle{{^horizontal}}-o{{/horizontal}}"></i> {{labels.horizontal}}</a>
</li>
<li {{#wide}}class="active"{{/wide}}>
<a href="javascript:void(0);" class="layout-wide"><i class="fa fa-circle{{^wide}}-o{{/wide}}"></i> {{labels.wide}}</a>
</li>
<li {{#vertical}}class="active"{{/vertical}}>
<a href="javascript:void(0);" class="layout-vertical"><i class="fa fa-circle{{^vertical}}-o{{/vertical}}"></i> {{labels.vertical}}</a>
</li>
</ul>
<h3><i class="fa fa-cog"></i> {{labels.file_opts}}</h3>
<ul>
<li><a href="javascript:void(0);" class="export-json"><i class="fa fa-code"></i> {{labels.export_json}}</a></li>
</ul>
</script>
<!-- TPL Chart tooltip -->
<script id="tpl-chart-tooltip" type="text/template">
<table class="chart-tooltip">
<tbody>
<tr>
<th colspan="2">{{data.0}}</th>
</tr>
<tr>
<td class="name"><span class="blue"></span>hits</td>
<td class="value">{{data.1}}</td>
</tr>
{{#data.2}}
<tr>
<td class="name"><span class="red"></span>visitors</td>
<td class="value">{{data.2}}</td>
</tr>
{{/data.2}}
</tbody>
</table>
</script>
snap/snapcraft.yaml
+28
View File
@@ -0,0 +1,28 @@
name: goaccess
version: git
summary: Real-time web log analyzer and interactive viewer
description: |
GoAccess is a real-time web log analyzer and interactive viewer that runs
in a terminal in *nix systems or through your browser.
confinement: classic
grade: devel
apps:
goaccess:
command: bin/goaccess
parts:
goaccess:
plugin: autotools
source: .
configflags:
- --enable-utf8
- --enable-geoip=legacy
build-packages:
- gettext
- libncurses-dev
- libncursesw5-dev
- libgeoip-dev
stage-packages:
- libgeoip-dev
src/base64.c
+243
View File
@@ -0,0 +1,243 @@
/**
* base64.c -- A basic base64 encode implementation
* _______ _______ __ __
* / ____/ | / / ___/____ _____/ /_____ / /_
* / / __ | | /| / /\__ \/ __ \/ ___/ //_/ _ \/ __/
* / /_/ / | |/ |/ /___/ / /_/ / /__/ ,< / __/ /_
* \____/ |__/|__//____/\____/\___/_/|_|\___/\__/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include <string.h>
#include <stdlib.h>
#include <inttypes.h>
#include "base64.h"
#include "xmalloc.h"
/* Encodes the given data with base64.
*
* On success, the encoded nul-terminated data, as a string is returned. */
char *
base64_encode (const void *buf, size_t size) {
static const char base64[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
char *str = (char *) xmalloc ((size + 3) * 4 / 3 + 1);
char *p = str;
const unsigned char *q = (const unsigned char *) buf;
size_t i = 0;
while (i < size) {
int c = q[i++];
c *= 256;
if (i < size)
c += q[i];
i++;
c *= 256;
if (i < size)
c += q[i];
i++;
*p++ = base64[(c & 0x00fc0000) >> 18];
*p++ = base64[(c & 0x0003f000) >> 12];
if (i > size + 1)
*p++ = '=';
else
*p++ = base64[(c & 0x00000fc0) >> 6];
if (i > size)
*p++ = '=';
else
*p++ = base64[c & 0x0000003f];
}
*p = 0;
return str;
}
/*
* base64_decode
*
* Given a Base64 encoded string in 'data', this function decodes it into
* a newly allocated binary buffer. The length of the decoded data is stored
* in *out_len. The caller is responsible for freeing the returned buffer.
*
* Returns NULL on error (for example, if the data's length is not a multiple
* of 4).
*/
char *
base64_decode (const char *data, size_t *out_len) {
size_t decoded_len = 0, i = 0, j = 0, pad = 0, len = 0;
char *out = NULL;
uint32_t triple = 0;
/* Create a lookup table for decoding.
* For valid Base64 characters 'A'-'Z', 'a'-'z', '0'-'9', '+', '/',
* we place their value; for '=' we simply return 0.
* All other characters are marked as 0x80 (an invalid marker).
*/
static const unsigned char dtable[256] = {
['A'] = 0,['B'] = 1,['C'] = 2,['D'] = 3,
['E'] = 4,['F'] = 5,['G'] = 6,['H'] = 7,
['I'] = 8,['J'] = 9,['K'] = 10,['L'] = 11,
['M'] = 12,['N'] = 13,['O'] = 14,['P'] = 15,
['Q'] = 16,['R'] = 17,['S'] = 18,['T'] = 19,
['U'] = 20,['V'] = 21,['W'] = 22,['X'] = 23,
['Y'] = 24,['Z'] = 25,
['a'] = 26,['b'] = 27,['c'] = 28,['d'] = 29,
['e'] = 30,['f'] = 31,['g'] = 32,['h'] = 33,
['i'] = 34,['j'] = 35,['k'] = 36,['l'] = 37,
['m'] = 38,['n'] = 39,['o'] = 40,['p'] = 41,
['q'] = 42,['r'] = 43,['s'] = 44,['t'] = 45,
['u'] = 46,['v'] = 47,['w'] = 48,['x'] = 49,
['y'] = 50,['z'] = 51,
['0'] = 52,['1'] = 53,['2'] = 54,['3'] = 55,
['4'] = 56,['5'] = 57,['6'] = 58,['7'] = 59,
['8'] = 60,['9'] = 61,
['+'] = 62,['/'] = 63,
['='] = 0,
/* All other values are implicitly 0 (or you can mark them as invalid
by setting them to 0x80 if you prefer stricter checking). */
};
len = strlen (data);
/* Validate length: Base64 encoded data must be a multiple of 4 */
if (len % 4 != 0)
return NULL;
/* Count padding characters at the end */
if (len) {
if (data[len - 1] == '=')
pad++;
if (len > 1 && data[len - 2] == '=')
pad++;
}
/* Calculate the length of the decoded data */
decoded_len = (len / 4) * 3 - pad;
out = (char *) xmalloc (decoded_len + 1); /* +1 for a null terminator if needed */
for (i = 0, j = 0; i < len;) {
unsigned int sextet_a = data[i] == '=' ? 0 : dtable[(unsigned char) data[i]];
unsigned int sextet_b = data[i + 1] == '=' ? 0 : dtable[(unsigned char) data[i + 1]];
unsigned int sextet_c = data[i + 2] == '=' ? 0 : dtable[(unsigned char) data[i + 2]];
unsigned int sextet_d = data[i + 3] == '=' ? 0 : dtable[(unsigned char) data[i + 3]];
i += 4;
triple = (sextet_a << 18) | (sextet_b << 12) | (sextet_c << 6) | sextet_d;
if (j < decoded_len)
out[j++] = (triple >> 16) & 0xFF;
if (j < decoded_len)
out[j++] = (triple >> 8) & 0xFF;
if (j < decoded_len)
out[j++] = triple & 0xFF;
}
out[decoded_len] = '\0'; /* Null-terminate the output buffer */
if (out_len)
*out_len = decoded_len;
return out;
}
/*
* base64UrlEncode - Converts a standard Base64 encoded string
* into Base64Url format.
*
* This replaces '+' with '-', '/' with '_', and removes '=' padding.
*/
char *
base64UrlEncode (const char *base64) {
char *url = NULL;
size_t len = 0;
if (!base64)
return NULL;
// Duplicate the input string to modify it in-place.
url = strdup (base64);
if (!url)
return NULL;
// Replace characters: '+' => '-', '/' => '_'
for (char *p = url; *p; p++) {
if (*p == '+')
*p = '-';
else if (*p == '/')
*p = '_';
}
// Remove trailing '=' characters.
len = strlen (url);
while (len && url[len - 1] == '=') {
url[len - 1] = '\0';
len--;
}
return url;
}
/*
* base64UrlDecode - Converts a Base64Url encoded string into standard Base64 format.
*
* This replaces '-' with '+', '_' with '/', and adds the necessary '=' padding.
* Caller is responsible for freeing the returned string.
*/
char *
base64UrlDecode (const char *base64Url) {
size_t len = 0, padding = 0, i = 0;
char *padded = NULL, *base64 = NULL;
if (!base64Url)
return NULL;
// Duplicate the input so we can modify it.
base64 = strdup (base64Url);
if (!base64)
return NULL;
// Replace '-' with '+' and '_' with '/'
for (char *p = base64; *p; p++) {
if (*p == '-')
*p = '+';
else if (*p == '_')
*p = '/';
}
// Calculate the padding required (base64 length must be a multiple of 4)
len = strlen (base64);
padding = (4 - (len % 4)) % 4;
padded = xmalloc (len + padding + 1);
if (!padded) {
free (base64);
return NULL;
}
strcpy (padded, base64);
for (i = 0; i < padding; i++) {
padded[len + i] = '=';
}
padded[len + padding] = '\0';
free (base64);
return padded;
}
src/base64.h
+40
View File
@@ -0,0 +1,40 @@
/**
* _______ _______ __ __
* / ____/ | / / ___/____ _____/ /_____ / /_
* / / __ | | /| / /\__ \/ __ \/ ___/ //_/ _ \/ __/
* / /_/ / | |/ |/ /___/ / /_/ / /__/ ,< / __/ /_
* \____/ |__/|__//____/\____/\___/_/|_|\___/\__/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef BASE64_H_INCLUDED
#define BASE64_H_INCLUDED
#include <stddef.h>
char *base64_encode (const void *buf, size_t size);
char *base64_decode (const char *data, size_t *out_len);
char *base64UrlDecode (const char *base64Url);
char *base64UrlEncode (const char *base64);
#endif // for #ifndef BASE64_H
src/bin2c.c
+154
View File
@@ -0,0 +1,154 @@
/*
* This is bin2c program, which allows you to convert binary file to
* C language array, for use as embedded resource, for instance you can
* embed graphics or audio file directly into your program.
* This is public domain software, use it on your own risk.
* Contact Serge Fukanchik at fuxx@mail.ru if you have any questions.
*
* Some modifications were made by Gwilym Kuiper (kuiper.gwilym@gmail.com)
* I have decided not to change the licence.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifdef USE_BZ2
#include <bzlib.h>
#endif
int
main (int argc, char *argv[]) {
char *buf;
char *ident;
int need_comma;
long file_size_orig;
size_t file_size, i;
FILE *f_input, *f_output;
#ifdef USE_BZ2
int status;
char *bz2_buf;
unsigned int uncompressed_size, bz2_size;
#endif
if (argc < 4) {
fprintf (stderr, "Usage: %s binary_file output_file array_name\n", argv[0]);
return -1;
}
f_input = fopen (argv[1], "rb");
if (f_input == NULL) {
fprintf (stderr, "%s: can't open %s for reading\n", argv[0], argv[1]);
return -1;
}
// Get the file length
fseek (f_input, 0, SEEK_END);
file_size_orig = ftell (f_input);
if (file_size_orig < 0) {
fprintf (stderr, "%s: can't get size of file %s\n", argv[0], argv[1]);
fclose (f_input);
return -1;
}
file_size = (size_t) file_size_orig;
fseek (f_input, 0, SEEK_SET);
if ((buf = malloc (file_size)) == NULL) {
fprintf (stderr, "Unable to malloc bin2c.c buffer\n");
fclose (f_input);
return -1;
}
if (fread (buf, file_size, 1, f_input) != 1) {
fprintf (stderr, "%s: can't read from %s\n", argv[0], argv[1]);
free (buf);
fclose (f_input);
return -1;
}
if (fgetc (f_input) != EOF) {
fprintf (stderr, "%s: can't read complete file %s\n", argv[0], argv[1]);
free (buf);
fclose (f_input);
return -1;
}
if (ferror (f_input)) {
fprintf (stderr, "%s: error while reading from %s\n", argv[0], argv[1]);
free (buf);
fclose (f_input);
return -1;
}
fclose (f_input);
#ifdef USE_BZ2
// allocate for bz2.
bz2_size = (file_size + file_size / 100 + 1) + 600; // as per the documentation
if ((bz2_buf = malloc (bz2_size)) == NULL) {
fprintf (stderr, "Unable to malloc bin2c.c buffer\n");
free (buf);
return -1;
}
// compress the data
status = BZ2_bzBuffToBuffCompress (bz2_buf, &bz2_size, buf, file_size, 9, 1, 0);
if (status != BZ_OK) {
fprintf (stderr, "Failed to compress data: error %i\n", status);
free (buf);
free (bz2_buf);
return -1;
}
// and be very lazy
free (buf);
uncompressed_size = file_size;
file_size = bz2_size;
buf = bz2_buf;
#endif
f_output = fopen (argv[2], "w");
if (f_output == NULL) {
fprintf (stderr, "%s: can't open %s for writing\n", argv[0], argv[1]);
free (buf);
return -1;
}
ident = argv[3];
need_comma = 0;
fprintf (f_output, "const char %s[%lu] = {", ident, file_size);
for (i = 0; i < file_size; ++i) {
if (buf[i] == '\0') {
fprintf (stderr,
"%s: writing a null character terminates the content prematurely\n", argv[0]);
fclose (f_output);
free (buf);
return -1;
}
if (need_comma)
fprintf (f_output, ", ");
else
need_comma = 1;
if ((i % 11) == 0)
fprintf (f_output, "\n\t");
fprintf (f_output, "0x%.2x", buf[i] & 0xff);
}
fprintf (f_output, "\n};\n\n");
fprintf (f_output, "const int %s_length = %lu;\n", ident, file_size);
#ifdef USE_BZ2
fprintf (f_output, "const int %s_length_uncompressed = %u;\n", ident, uncompressed_size);
#endif
if (ferror (f_output)) {
fprintf (stderr, "%s: error while writing to %s\n", argv[0], argv[2]);
fclose (f_output);
free (buf);
return -1;
}
fclose (f_output);
free (buf);
return 0;
}
src/bitmap.c
+173
View File
@@ -0,0 +1,173 @@
/**
* bitmap.c -- A quick bitmap implementation
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#include "error.h"
#include "xmalloc.h"
#include "bitmap.h"
/*
* Returns the hamming weight (i.e. the number of bits set) in a word.
* NOTE: This routine borrowed from Linux 2.4.9 <linux/bitops.h>.
*/
static uint32_t
hweight (uint32_t w) {
uint32_t res;
res = (w & 0x55555555) + ((w >> 1) & 0x55555555);
res = (res & 0x33333333) + ((res >> 2) & 0x33333333);
res = (res & 0x0F0F0F0F) + ((res >> 4) & 0x0F0F0F0F);
res = (res & 0x00FF00FF) + ((res >> 8) & 0x00FF00FF);
res = (res & 0x0000FFFF) + ((res >> 16) & 0x0000FFFF);
return res;
}
/* Returns the number of bytes required for bm->len bits. */
inline uint32_t
bitmap_sizeof (uint32_t i) {
return bitmap_word (i) * sizeof (word_t);
}
void
free_bitmap (bitmap *bm) {
free ((uint32_t *) bm->bmp);
free (bm);
}
int
bitmap_set_bit (word_t *words, uint32_t n) {
words[WORD_OFFSET (n)] |= ((word_t) 1 << BIT_OFFSET (n));
return 0;
}
int
bitmap_get_bit (word_t *words, uint32_t n) {
word_t bit = words[WORD_OFFSET (n)] & ((word_t) 1 << BIT_OFFSET (n));
return bit != 0;
}
uint32_t
bitmap_count_set (const bitmap *bm) {
uint32_t i, n = 0, len = 0;
if (!bm)
return 0;
len = bitmap_word (bm->len);
for (i = 0; i < len; ++i)
n += hweight (bm->bmp[i]);
return n;
}
uint32_t
bitmap_ffs (bitmap *bm) {
uint32_t i, pos = 1, len = 0;
uint32_t __bitset;
if (!bm)
return 0;
len = bitmap_word (bm->len);
for (i = 0; i < len; ++i) {
__bitset = bm->bmp[i];
if ((pos = __builtin_ffs (__bitset)))
break;
}
return (BITS_PER_WORD * i) + pos;
}
bitmap *
bitmap_create (uint32_t bit) {
bitmap *bm = xcalloc (1, sizeof (bitmap));
uint32_t bytes = bitmap_sizeof (bit);
bm->bmp = xmalloc (bytes);
memset (bm->bmp, 0, bytes);
bm->len = bit;
return bm;
}
int
bitmap_realloc (bitmap *bm, uint32_t bit) {
uint32_t *tmp = NULL;
uint32_t oldlen = 0, newlen = 0;
newlen = bitmap_sizeof (bit);
oldlen = bitmap_sizeof (bm->len);
if (newlen <= oldlen)
return 1;
tmp = realloc (bm->bmp, newlen);
if ((tmp == NULL && newlen > 0) || bit < bm->len)
FATAL ("Unable to realloc bitmap hash value %u %u", newlen, bm->len);
LOG_DEBUG (("bit: %d, bm->len: %d, oldlen: %d, newlen: %d\n", bit, bm->len, oldlen, newlen));
memset (tmp + bitmap_word (bm->len), 0, (newlen - oldlen));
bm->len = bit;
bm->bmp = tmp;
return 0;
}
bitmap *
bitmap_copy (const bitmap *bm) {
bitmap *ret;
if (!bm)
return NULL;
if (!(ret = bitmap_create (bm->len)))
return NULL;
memcpy (ret->bmp, bm->bmp, bitmap_sizeof (bm->len));
return ret;
}
int
bitmap_key_exists (bitmap *bm, uint32_t bit) {
if (bm->len < bit)
bitmap_realloc (bm, bit);
/* if bit set, then it's the same visitor */
if (bitmap_get_bit (bm->bmp, bit - 1))
return 1;
bitmap_set_bit (bm->bmp, bit - 1);
return 0;
}
src/bitmap.h
+76
View File
@@ -0,0 +1,76 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef BITMAP_H_INCLUDED
#define BITMAP_H_INCLUDED
#include <limits.h> /* for CHAR_BIT */
typedef uint32_t word_t; // I want to change this, from uint32_t to uint64_t
enum { BITS_PER_WORD = sizeof (word_t) * CHAR_BIT };
#define WORD_OFFSET(b) ((b) / BITS_PER_WORD)
#define BIT_OFFSET(b) ((b) % BITS_PER_WORD)
typedef struct bitmap_ {
uint32_t *bmp;
uint32_t len; /** Length of the bitmap, in bits */
} bitmap;
static inline uint32_t
bitmap_word (uint32_t i) {
return (((i) + (BITS_PER_WORD) - 1) / (BITS_PER_WORD));
}
#define BITMAP_FOREACH(bm, pos, code) { size_t __k; \
int __r; uint32_t __bitset, __t; \
uint32_t __len = bitmap_word(bm->len); \
for (__k = 0; __k < __len; ++__k) { \
__bitset = bm->bmp[__k]; \
while (__bitset != 0) { \
__t = __bitset & -__bitset; \
__r = __builtin_ctzl (__bitset); \
pos = (__k * 32) + __r + 1; \
code; \
__bitset ^= __t; \
} \
} }
bitmap *bitmap_copy (const bitmap * bm);
bitmap *bitmap_create (uint32_t bit);
int bitmap_get_bit (word_t * words, uint32_t n);
int bitmap_key_exists (bitmap * bm, uint32_t bit);
int bitmap_realloc (bitmap * bm, uint32_t bit);
int bitmap_set_bit (word_t * words, uint32_t n);
uint32_t bitmap_count_set (const bitmap * bm);
uint32_t bitmap_ffs (bitmap * bm);
uint32_t bitmap_sizeof (uint32_t nbits);
void free_bitmap (bitmap * bm);
#endif // for #ifndef BITMAP_H
src/browsers.c
+658
View File
@@ -0,0 +1,658 @@
/**
6 browsers.c -- functions for dealing with browsers
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stddef.h>
#include "browsers.h"
#include "error.h"
#include "settings.h"
#include "util.h"
#include "xmalloc.h"
/* ###NOTE: The size of the list is proportional to the run time,
* which makes this pretty slow */
static char ***browsers_hash = NULL;
static const char *const browsers[][2] = {
/* Game Systems: place game console browsers here */
{"Xbox One", "Game Systems"},
{"Xbox", "Game Systems"},
{"PlayStation", "Game Systems"},
{"NintendoBrowser", "Game Systems"},
{"Valve Steam", "Game Systems"},
{"Origin", "Game Systems"},
{"Raptr", "Game Systems"},
/* Based on Internet Explorer: browsers based on IE engine */
{"Avant Browser", "Others"},
{"IEMobile", "MSIE"},
{"MSIE", "MSIE"},
{"Trident/7.0", "MSIE"},
{"Edg", "Edge"},
{"Edge", "Edge"},
/* Surf Browser */
{"Surf", "Surf"},
/* Opera: place all opera-related user agents here */
{"Opera Mini", "Opera"},
{"Opera Mobi", "Opera"},
{"Opera", "Opera"},
{"OPR", "Opera"},
{"OPiOS", "Opera"},
{"Coast", "Opera"},
{"FBAV", "Facebook App"},
{"FBSV", "Facebook App"},
{"FBAN", "Facebook App"},
{"FB_IAB", "Facebook App"},
/* Others: miscellaneous user agents that don't fit other categories */
{"Homebrew", "Others"},
{"APT-", "Others"},
{"Apt-Cacher", "Others"},
{"Aptly", "Others"},
{"Chef Client", "Others"},
{"Huawei", "Others"},
{"HUAWEI", "Others"},
{"BlackBerry", "Others"},
{"BrowserX", "Others"},
{"Dalvik", "Others"},
{"Dillo", "Others"},
{"ELinks", "Others"},
{"Epiphany", "Others"},
{"Firebird", "Others"},
{"Galeon", "Others"},
{"google-cloud-sdk", "Others"},
{"IBrowse", "Others"},
{"K-Meleon", "Others"},
{"Konqueror", "Others"},
{"Links", "Others"},
{"Lynx", "Others"},
{"Midori", "Others"},
{"Minefield", "Others"},
{"Mosaic", "Others"},
{"Netscape", "Others"},
{"SeaMonkey", "Others"},
{"UCBrowser", "Others"},
{"Wget", "Others"},
{"libfetch", "Others"},
{"check_http", "Others"},
{"Go-http-client", "Others"},
{"curl", "Others"},
{"midori", "Others"},
{"w3m", "Others"},
{"MicroMessenger", "Others"},
{"Apache", "Others"},
{"JOSM", "Others"},
{"pacman", "Others"},
{"Pamac", "Others"},
{"libwww-perl", "Others"},
{"python-requests", "Others"},
{"PackageKit", "Others"},
{"F-Droid", "Others"},
{"okhttp", "Others"},
{"node", "Others"},
{"PrivacyBrowser", "Others"},
{"Transmission", "Others"},
{"libmpv", "Others"},
{"aria2", "Others"},
{"Electron", "Others"},
{"Chromium", "Chrome"},
{"QtWebEngine", "Others"},
{"SamsungBrowser", "Samsung Internet"},
{"Silk/", "Silk"},
{"MiuiBrowser", "Others"},
{"HuaweiBrowser", "Others"},
{"HeyTapBrowser", "Others"},
{"VivoBrowser", "Others"},
{"QQBrowser", "Others"},
{"Instagram", "Others"},
{"TikTok", "Others"},
{"Snapchat", "Others"},
{"Pinterest", "Others"},
{"Roku", "Others"},
{"Tizen", "Others"},
{"Web0S", "Others"},
/* Feed Readers: place all feed reader user agents here */
{"AppleNewsBot", "Feeds"},
{"Bloglines", "Feeds"},
{"Digg Feed Fetcher", "Feeds"},
{"Feedbin", "Feeds"},
{"FeedHQ", "Feeds"},
{"Feedly", "Feeds"},
{"Flipboard", "Feeds"},
{"inoreader.com", "Feeds"},
{"Netvibes", "Feeds"},
{"NewsBlur", "Feeds"},
{"PinRSS", "Feeds"},
{"theoldreader.com", "Feeds"},
{"WordPress.com Reader", "Feeds"},
{"YandexBlogs", "Feeds"},
{"Brainstorm", "Feeds"},
{"Mastodon", "Feeds"},
{"Pleroma", "Feeds"},
{"Akregator", "Feeds"},
{"Apple-PubSub", "Feeds"},
{"BTWebClient", "Feeds"},
{"com apple.Safari.WebFeedParser", "Feeds"},
{"FeedDemon", "Feeds"},
{"Feedy", "Feeds"},
{"Fever", "Feeds"},
{"FreshRSS", "Feeds"},
{"Liferea", "Feeds"},
{"NetNewsWire", "Feeds"},
{"RSSOwl", "Feeds"},
{"Thunderbird", "Feeds"},
{"Tiny Tiny RSS", "Feeds"},
{"Winds", "Feeds"},
/* Crawlers: all bot and crawler user agents grouped together */
{"AdsBot", "Crawlers"},
{"AdsBot-Google", "Crawlers"},
{"APIs-Google", "Crawlers"},
{"AppEngine-Google", "Crawlers"},
{"Mediapartners-Google", "Crawlers"},
{"Google-InspectionTool", "Crawlers"},
{"Google-Extended", "Crawlers"},
{"GoogleOther", "Crawlers"},
{"Google-Read-Aloud", "Crawlers"},
{"Google", "Crawlers"},
{"WhatsApp", "Crawlers"},
{"AppleBot", "Crawlers"},
{"Applebot-Extended", "Crawlers"},
{"facebookexternalhit", "Crawlers"},
{"Facebot", "Crawlers"},
{"Twitter", "Crawlers"},
{"Twitterbot", "Crawlers"},
{"Slackbot-LinkExpanding", "Crawlers"},
{"Slack-ImgProxy", "Crawlers"},
{"Slack", "Crawlers"},
{"TelegramBot", "Crawlers"},
{"Sogou", "Crawlers"},
{"Java", "Crawlers"},
{"Jakarta Commons-HttpClient", "Crawlers"},
{"netEstate", "Crawlers"},
{"PiplBot", "Crawlers"},
{"Baidu", "Crawlers"},
{"Mastodon", "Crawlers"},
{"IstellaBot", "Crawlers"},
{"heritrix", "Crawlers"},
{"PagesInventory", "Crawlers"},
{"rogerbot", "Crawlers"},
{"fastbot", "Crawlers"},
{"yacybot", "Crawlers"},
{"PycURL", "Crawlers"},
{"PHP", "Crawlers"},
{"ClaudeBot", "Crawlers"},
{"Claude-User", "Crawlers"},
{"AndroidDownloadManager", "Crawlers"},
{"Embedly", "Crawlers"},
{"ruby", "Crawlers"},
{"Ruby", "Crawlers"},
{"python", "Crawlers"},
{"Python", "Crawlers"},
{"LinkedIn", "Crawlers"},
{"LinkedInBot", "Crawlers"},
{"Microsoft-WebDAV", "Crawlers"},
{"DuckDuckBot", "Crawlers"},
{"bingbot", "Crawlers"},
{"PetalBot", "Crawlers"},
{"Discordbot", "Crawlers"},
{"ZoominfoBot", "Crawlers"},
{"Googlebot", "Crawlers"},
{"DotBot", "Crawlers"},
{"AhrefsBot", "Crawlers"},
{"SemrushBot", "Crawlers"},
{"archive.org_bot", "Crawlers"},
{"BLEXBot", "Crawlers"},
{"NetcraftSurveyAgent", "Crawlers"},
{"Netcraft Web Server Survey", "Crawlers"},
{"masscan", "Crawlers"},
{"MJ12bot", "Crawlers"},
{"Pandalytics", "Crawlers"},
{"YandexBot", "Crawlers"},
{"Nimbostratus-Bot", "Crawlers"},
{"HTTP Banner Detection", "Crawlers"},
{"Hakai", "Crawlers"},
{"WinHttp_WINHTTPREQUEST5", "Crawlers"},
{"NetSystemsResearch", "Crawlers"},
{"Nextcloud Server Crawler", "Crawlers"},
{"CFNetwork", "Crawlers"},
{"GoScraper", "Crawlers"},
{"Googlebot-Image", "Crawlers"},
{"ZmEu", "Crawlers"},
{"DowntimeDetector", "Crawlers"},
{"MauiBot", "Crawlers"},
{"stagefright", "Crawlers"},
{"ImagesiftBot", "Crawlers"},
{"Bytespider", "Crawlers"},
{"WordPress/", "Crawlers"},
{"Synapse", "Crawlers"},
{"Yahoo! Slurp", "Crawlers"},
{"GoogleMobile", "Crawlers"},
{"ZoteroTranslationServer", "Crawlers"},
{"OAI-SearchBot", "Crawlers"},
{"GPTBot", "Crawlers"},
{"PerplexityBot", "Crawlers"},
{"Amazonbot", "Crawlers"},
{"SkypeUriPreview", "Crawlers"},
{"OAI-SearchBot", "Crawlers"},
{"OAI-SearchBot", "Crawlers"}, /* duplicate-safe if you dedupe elsewhere */
/* Based on Firefox: place all Firefox-based browsers here */
{"Camino", "Others"},
{"Iceweasel", "Firefox"},
{"Waterfox", "Firefox"},
{"PaleMoon", "Firefox"},
{"Focus", "Firefox"},
{"Klar", "Firefox"},
{"Zen/", "Zen"},
{"Firefox", "Firefox"},
/* Based on Chromium: place all Chromium-based browsers here, including BeakerBrowser under Others for consistency */
{"Arc/", "Arc"},
{"Brave", "Brave"},
{"Vivaldi", "Vivaldi"},
{"YaBrowser", "Yandex"},
{"DuckDuckGo/", "DuckDuckGo Browser"},
/* Chrome: place all Chrome-related user agents here */
{"HeadlessChrome", "Chrome"},
{"Chrome", "Chrome"},
{"CriOS", "Chrome"},
/* Safari: place all Safari-related user agents here */
{"Safari", "Safari"},
/* HTTP Library or HTTP Server User Agents: place here */
{"axios", "HTTP Library"}, /* NodeJS axios */
{"lua-resty-http", "HTTP Library"}, /* Ngx luarestyhttp module */
{"Apache-HttpClient", "HTTP Library"},
{"GuzzleHttp", "HTTP Library"},
{"RestSharp", "HTTP Library"},
{"http.rb", "HTTP Library"},
{"Java-http-client", "HTTP Library"},
{"node-fetch", "HTTP Library"},
/* Citation Services: place here */
{"Citoid", "Citation"}, /* MediaWiki Citoid */
{"EasyBib", "Citation"}, /* Easybib */
/* Podcast Fetchers: place here */
{"Downcast", "Podcasts"},
{"gPodder", "Podcasts"},
{"Instacast", "Podcasts"},
{"iTunes", "Podcasts"},
{"Miro", "Podcasts"},
{"Pocket Casts", "Podcasts"},
{"BashPodder", "Podcasts"},
/* Uptime and Monitoring Clients: place here */
{"Pingdom.com", "Uptime"},
{"jetmon", "Uptime"},
{"NodeUptime", "Uptime"},
{"NewRelicPinger", "Uptime"},
{"StatusCake", "Uptime"},
{"internetVista", "Uptime"},
{"Server Density Service Monitoring v2", "Uptime"},
{"Better Uptime Bot", "Uptime"},
{"Site24x7", "Uptime"},
{"Uptime-Kuma", "Uptime"},
{"UptimeRobot", "Uptime"},
{"NewRelicSynthetics", "Uptime"},
{"DatadogSynthetics", "Uptime"},
{"ELB-HealthChecker", "Uptime"},
{"GoogleHC", "Uptime"},
{"kube-probe", "Uptime"},
{"Fastly-HealthChecker", "Uptime"},
{"Cloudflare-Healthchecks", "Uptime"},
{"CloudFlare-AlwaysOnline", "Uptime"},
/* Performance and Caching: place here */
{"ShortPixel", "Performance"}, /* Image Optimization */
{"WP Rocket", "Caching"}, /* Preloading Cache for WordPress Plugin */
/* Security: place here */
{"Barracuda Sentinel", "Security"}, /* Barricuda spear fishing service */
{"ACI Site Scanner", "Security"}, /* Can't confirm specific vendor */
/* Mozilla: catch-all for Mozilla-related user agents */
{"mozilla", "Others"}
};
/* Free all browser entries from our array of key/value pairs. */
void
free_browsers_hash (void) {
size_t i;
int j;
for (i = 0; i < ARRAY_SIZE (browsers); ++i) {
free (browsers_hash[i][0]);
free (browsers_hash[i][1]);
free (browsers_hash[i]);
}
free (browsers_hash);
for (j = 0; j < conf.browsers_hash_idx; ++j) {
free (conf.user_browsers_hash[j][0]);
free (conf.user_browsers_hash[j][1]);
free (conf.user_browsers_hash[j]);
}
if (conf.browsers_file) {
free (conf.user_browsers_hash);
}
}
static int
is_dup (char ***list, int len, const char *browser) {
int i;
/* check for dups */
for (i = 0; i < len; ++i) {
if (strcmp (browser, list[i][0]) == 0)
return 1;
}
return 0;
}
/* Set a browser/type pair into our multidimensional array of browsers.
*
* On duplicate functions returns void.
* Otherwise memory is mallo'd for our array entry. */
static void
set_browser (char ***list, int idx, const char *browser, const char *type) {
list[idx] = xcalloc (2, sizeof (char *));
list[idx][0] = xstrdup (browser);
list[idx][1] = xstrdup (type);
}
/* Parse the key/value pair from the browser list file. */
static void
parse_browser_token (char ***list, char *line, int n) {
char *val;
size_t idx = 0;
/* key */
idx = strcspn (line, "\t");
if (strlen (line) == idx)
FATAL ("Malformed browser name at line: %d", n);
line[idx] = '\0';
/* value */
val = line + (idx + 1);
idx = strspn (val, "\t");
if (strlen (val) == idx)
FATAL ("Malformed browser category at line: %d", n);
val = val + idx;
val = trim_str (val);
if (is_dup (list, conf.browsers_hash_idx, line)) {
LOG_INVALID (("Duplicate browser entry: %s", line));
return;
}
set_browser (list, conf.browsers_hash_idx, line, val);
conf.browsers_hash_idx++;
}
/* Parse our default array of browsers and put them on our hash including those
* from the custom parsed browsers file.
*
* On error functions returns void.
* Otherwise browser entries are put into the hash. */
void
parse_browsers_file (void) {
char line[MAX_LINE_BROWSERS + 1];
FILE *file;
int n = 0;
size_t i, len = ARRAY_SIZE (browsers);
browsers_hash = xmalloc (ARRAY_SIZE (browsers) * sizeof (char **));
/* load hash from the browser's array (default) */
for (i = 0; i < len; ++i) {
set_browser (browsers_hash, i, browsers[i][0], browsers[i][1]);
}
if (!conf.browsers_file)
return;
/* could not open browsers file */
if ((file = fopen (conf.browsers_file, "r")) == NULL)
FATAL ("Unable to open browser's file: %s", strerror (errno));
conf.user_browsers_hash = xmalloc (MAX_CUSTOM_BROWSERS * sizeof (char **));
/* load hash from the user's given browsers file */
while (fgets (line, sizeof line, file) != NULL) {
while (line[0] == ' ' || line[0] == '\t')
memmove (line, line + 1, strlen (line));
n++;
if (line[0] == '\n' || line[0] == '\r' || line[0] == '#')
continue;
if (conf.browsers_hash_idx >= MAX_CUSTOM_BROWSERS)
FATAL ("Maximum number of custom browsers has been reached");
parse_browser_token (conf.user_browsers_hash, line, n);
}
fclose (file);
}
/* Determine if the user-agent is a crawler.
*
* On error or is not a crawler, 0 is returned.
* If it is a crawler, 1 is returned . */
int
is_crawler (const char *agent) {
char btype[BROWSER_TYPE_LEN];
char *browser, *a;
if (agent == NULL || *agent == '\0')
return 0;
if ((a = xstrdup (agent), browser = verify_browser (a, btype)) != NULL)
free (browser);
free (a);
return strcmp (btype, "Crawlers") == 0 ? 1 : 0;
}
/* Return the Opera 15 and beyond.
*
* On success, the opera string and version is returned. */
static char *
parse_opera (char *token) {
char *val = xmalloc (snprintf (NULL, 0, "Opera%s", token) + 1);
sprintf (val, "Opera%s", token);
return val;
}
/* Given the original user agent string, and a partial crawler match, iterate
* back until the next delimiter is found and return occurrence.
*
* On error when attempting to extract crawler, NULL is returned.
* If a possible crawler string is matched, then possible bot is returned . */
static char *
parse_crawler (char *str, char *match, char *type) {
char *ptr = NULL;
int found = 0;
while (match != str) {
match--;
if (*match == ' ' || *match == '+' || match == str) {
found = 1;
break;
}
}
/* same addr */
if (!found && match == str)
return NULL;
/* account for the previous +|space */
if (found && match != str)
match++;
if ((ptr = strpbrk (match, "; ")))
*ptr = '\0';
/* empty string after parsing it */
if (*match == '\0')
return NULL;
xstrncpy (type, "Crawlers", BROWSER_TYPE_LEN);
return xstrdup (match);
}
/* If the following string matches are found within user agent, then it's
* highly likely it's a possible crawler.
* Note that this could certainly return false positives.
*
* If no occurrences are found, NULL is returned.
* If an occurrence is found, a pointer to the match is returned . */
static char *
check_http_crawler (const char *str) {
char *match = NULL;
/* e.g., compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm */
if ((match = strstr (str, "; +http")))
return match;
/* compatible; UptimeRobot/2.0; http://www.uptimerobot.com/ */
if ((match = strstr (str, "; http")))
return match;
/* Slack-ImgProxy (+https://api.slack.com/robots) */
if ((match = strstr (str, " (+http")))
return match;
/* TurnitinBot/3.0 (http://www.turnitin.com/robot/crawlerinfo.html) */
if ((match = strstr (str, " (http")))
return match;
/* w3c e.g., (compatible;+Googlebot/2.1;++http://www.google.com/bot.html) */
if ((match = strstr (str, ";++http")))
return match;
return NULL;
}
/* Parse the given user agent match and extract the browser string.
*
* If no match, the original match is returned.
* Otherwise the parsed browser is returned. */
static char *
parse_browser (char *match, char *type, int i, char ***hash) {
char *b = NULL, *ptr = NULL, *slh = NULL;
size_t cnt = 0, space = 0;
match = char_replace (match, '+', '-');
/* Check if there are spaces in the token string, that way strpbrk
* does not stop at the first space within the token string */
if ((cnt = count_matches (hash[i][0], ' ')) && (b = match)) {
while (space++ < cnt && (b = strchr (b, ' ')))
b++;
} else
b = match;
xstrncpy (type, hash[i][1], BROWSER_TYPE_LEN);
/* Internet Explorer 11 */
if (strstr (match, "rv:11") && strstr (match, "Trident/7.0")) {
return alloc_string ("MSIE/11.0");
}
/* Opera +15 uses OPR/# */
if (strstr (match, "OPR") != NULL && (slh = strrchr (match, '/'))) {
return parse_opera (slh);
}
/* Opera has the version number at the end */
if (strstr (match, "Opera") && (slh = strrchr (match, '/')) && match < slh) {
memmove (match + 5, slh, strlen (slh) + 1);
}
/* IE Old */
if (strstr (match, "MSIE") != NULL) {
if ((ptr = strpbrk (match, ";)-")) != NULL)
*ptr = '\0';
match = char_replace (match, ' ', '/');
}
/* all others */
else if ((ptr = strpbrk (b ? b : match, ";) \\")) != NULL) {
*ptr = '\0';
}
return alloc_string (match);
}
/* Given a user agent, determine the browser used.
*
* ###NOTE: The size of the list is proportional to the run time,
* which makes this pretty slow
*
* On error, NULL is returned.
* On success, a malloc'd string containing the browser is returned. */
char *
verify_browser (char *str, char *type) {
char *match = NULL, *token = NULL;
int i = 0;
size_t j = 0;
if (str == NULL || *str == '\0')
return NULL;
/* check user's list */
for (i = 0; i < conf.browsers_hash_idx; ++i) {
if ((match = strstr (str, conf.user_browsers_hash[i][0])) == NULL)
continue;
return parse_browser (match, type, i, conf.user_browsers_hash);
}
/* fallback to default browser list */
for (j = 0; j < ARRAY_SIZE (browsers); ++j) {
if ((match = strstr (str, browsers_hash[j][0])) == NULL)
continue;
return parse_browser (match, type, j, browsers_hash);
}
/* try heuristics */
if ((match = check_http_crawler (str)) && (token = parse_crawler (str, match, type)))
return token;
if (conf.unknowns_log)
LOG_UNKNOWNS (("%-7s%s\n", "[BR]", str));
if (conf.unknowns_as_crawlers && strcmp (type, "Crawlers"))
xstrncpy (type, "Crawlers", BROWSER_TYPE_LEN);
else
xstrncpy (type, "Unknown", BROWSER_TYPE_LEN);
return alloc_string ("Unknown");
}
src/browsers.h
+48
View File
@@ -0,0 +1,48 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef BROWSERS_H_INCLUDED
#define BROWSERS_H_INCLUDED
#define BROWSER_TYPE_LEN 24
#define MAX_LINE_BROWSERS 128
#define MAX_CUSTOM_BROWSERS 256
/* Each Browser contains the number of hits and the Browser's type */
typedef struct GBrowser_ {
char browser_type[BROWSER_TYPE_LEN];
int hits;
} GBrowser;
char *verify_browser (char *str, char *browser_type);
int is_crawler (const char *agent);
void free_browsers_hash (void);
void parse_browsers_file (void);
#endif
src/color.c
+841
View File
@@ -0,0 +1,841 @@
/**
* color.c -- functions related to custom color
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
#include "color.h"
#include "error.h"
#include "gslist.h"
#include "util.h"
#include "xmalloc.h"
static GSLList *color_list = NULL;
static GSLList *pair_list = NULL;
/* *INDENT-OFF* */
static const GEnum CSTM_COLORS[] = {
{"COLOR_MTRC_HITS" , COLOR_MTRC_HITS},
{"COLOR_MTRC_VISITORS" , COLOR_MTRC_VISITORS},
{"COLOR_MTRC_HITS_PERC" , COLOR_MTRC_HITS_PERC},
{"COLOR_MTRC_VISITORS_PERC" , COLOR_MTRC_VISITORS_PERC},
{"COLOR_MTRC_BW_PERC" , COLOR_MTRC_BW_PERC},
{"COLOR_MTRC_BW" , COLOR_MTRC_BW},
{"COLOR_MTRC_AVGTS" , COLOR_MTRC_AVGTS},
{"COLOR_MTRC_CUMTS" , COLOR_MTRC_CUMTS},
{"COLOR_MTRC_MAXTS" , COLOR_MTRC_MAXTS},
{"COLOR_MTRC_PROT" , COLOR_MTRC_PROT},
{"COLOR_MTRC_MTHD" , COLOR_MTRC_MTHD},
{"COLOR_MTRC_DATA" , COLOR_MTRC_DATA},
{"COLOR_MTRC_HITS_PERC_MAX" , COLOR_MTRC_HITS_PERC_MAX},
{"COLOR_MTRC_VISITORS_PERC_MAX" , COLOR_MTRC_VISITORS_PERC_MAX},
{"COLOR_MTRC_BW_PERC_MAX" , COLOR_MTRC_BW_PERC_MAX},
{"COLOR_PANEL_COLS" , COLOR_PANEL_COLS},
{"COLOR_BARS" , COLOR_BARS},
{"COLOR_SUBBARS" , COLOR_SUBBARS},
{"COLOR_CHART_AXIS" , COLOR_CHART_AXIS},
{"COLOR_CHART_VALUES" , COLOR_CHART_VALUES},
{"COLOR_ERROR" , COLOR_ERROR},
{"COLOR_SELECTED" , COLOR_SELECTED},
{"COLOR_PANEL_ACTIVE" , COLOR_PANEL_ACTIVE},
{"COLOR_PANEL_HEADER" , COLOR_PANEL_HEADER},
{"COLOR_PANEL_DESC" , COLOR_PANEL_DESC},
{"COLOR_OVERALL_LBLS" , COLOR_OVERALL_LBLS},
{"COLOR_OVERALL_VALS" , COLOR_OVERALL_VALS},
{"COLOR_OVERALL_PATH" , COLOR_OVERALL_PATH},
{"COLOR_ACTIVE_LABEL" , COLOR_ACTIVE_LABEL},
{"COLOR_BG" , COLOR_BG},
{"COLOR_DEFAULT" , COLOR_DEFAULT},
{"COLOR_PROGRESS" , COLOR_PROGRESS},
};
static const char *const colors256_mono[] = {
"COLOR_MTRC_HITS color7:color-1",
"COLOR_MTRC_VISITORS color8:color-1",
"COLOR_MTRC_DATA color7:color-1",
"COLOR_MTRC_BW color8:color-1",
"COLOR_MTRC_AVGTS color8:color-1",
"COLOR_MTRC_CUMTS color8:color-1",
"COLOR_MTRC_MAXTS color8:color-1",
"COLOR_MTRC_PROT color8:color-1",
"COLOR_MTRC_MTHD color7:color-1",
"COLOR_MTRC_HITS_PERC color0:color-1 bold",
"COLOR_MTRC_HITS_PERC color1:color-1 bold VISITORS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold OS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold BROWSERS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold VISIT_TIMES",
"COLOR_MTRC_HITS_PERC_MAX color0:color-1 bold",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold VISITORS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold OS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold BROWSERS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold VISIT_TIMES",
"COLOR_MTRC_VISITORS_PERC color0:color-1 bold",
"COLOR_MTRC_VISITORS_PERC_MAX color0:color-1 bold",
"COLOR_MTRC_BW_PERC color0:color-1 bold",
"COLOR_MTRC_BW_PERC_MAX color0:color-1 bold",
"COLOR_PANEL_COLS color7:color-1",
"COLOR_BARS color7:color-1",
"COLOR_SUBBARS color8:color-1",
"COLOR_CHART_AXIS color7:color-1",
"COLOR_CHART_VALUES color7:color-1",
"COLOR_ERROR color7:color1",
"COLOR_SELECTED color7:color8",
"COLOR_PANEL_ACTIVE color0:color3",
"COLOR_PANEL_HEADER color0:color7",
"COLOR_PANEL_DESC color7:color-1",
"COLOR_OVERALL_LBLS color7:color-1 bold",
"COLOR_OVERALL_VALS color6:color-1 bold",
"COLOR_OVERALL_PATH color3:color-1",
"COLOR_ACTIVE_LABEL color4:color7",
"COLOR_BG color7:color-1",
"COLOR_DEFAULT color7:color-1",
"COLOR_PROGRESS color0:color6",
};
static const char *const colors256_green[] = {
"COLOR_MTRC_HITS color7:color-1",
"COLOR_MTRC_VISITORS color8:color-1",
"COLOR_MTRC_DATA color7:color-1",
"COLOR_MTRC_BW color8:color-1",
"COLOR_MTRC_AVGTS color8:color-1",
"COLOR_MTRC_CUMTS color8:color-1",
"COLOR_MTRC_MAXTS color8:color-1",
"COLOR_MTRC_PROT color8:color-1",
"COLOR_MTRC_MTHD color7:color-1",
"COLOR_MTRC_HITS_PERC color0:color-1 bold",
"COLOR_MTRC_HITS_PERC color1:color-1 bold VISITORS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold OS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold BROWSERS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold VISIT_TIMES",
"COLOR_MTRC_HITS_PERC_MAX color0:color-1 bold",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold VISITORS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold OS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold BROWSERS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold VISIT_TIMES",
"COLOR_MTRC_VISITORS_PERC color0:color-1 bold",
"COLOR_MTRC_VISITORS_PERC_MAX color0:color-1 bold",
"COLOR_MTRC_BW_PERC color0:color-1 bold",
"COLOR_MTRC_BW_PERC_MAX color0:color-1 bold",
"COLOR_PANEL_COLS color7:color-1",
"COLOR_BARS color7:color-1",
"COLOR_SUBBARS color8:color-1",
"COLOR_CHART_AXIS color7:color-1",
"COLOR_CHART_VALUES color7:color-1",
"COLOR_ERROR color7:color1",
"COLOR_SELECTED color7:color8",
"COLOR_PANEL_ACTIVE color0:color3",
"COLOR_PANEL_HEADER color0:color35",
"COLOR_PANEL_DESC color7:color-1",
"COLOR_OVERALL_LBLS color7:color-1 bold",
"COLOR_OVERALL_VALS color6:color-1 bold",
"COLOR_OVERALL_PATH color3:color-1",
"COLOR_ACTIVE_LABEL color7:color35",
"COLOR_BG color7:color-1",
"COLOR_DEFAULT color7:color-1",
"COLOR_PROGRESS color0:color6",
};
static const char *const colors256_monokai[] = {
"COLOR_MTRC_HITS color197:color-1",
"COLOR_MTRC_VISITORS color148:color-1",
"COLOR_MTRC_DATA color7:color-1",
"COLOR_MTRC_BW color81:color-1",
"COLOR_MTRC_AVGTS color247:color-1",
"COLOR_MTRC_CUMTS color95:color-1",
"COLOR_MTRC_MAXTS color186:color-1",
"COLOR_MTRC_PROT color141:color-1",
"COLOR_MTRC_MTHD color81:color-1",
"COLOR_MTRC_HITS_PERC color186:color-1",
"COLOR_MTRC_HITS_PERC color186:color-1 VISITORS",
"COLOR_MTRC_HITS_PERC color186:color-1 OS",
"COLOR_MTRC_HITS_PERC color186:color-1 BROWSERS",
"COLOR_MTRC_HITS_PERC color186:color-1 VISIT_TIMES",
"COLOR_MTRC_HITS_PERC_MAX color208:color-1",
"COLOR_MTRC_HITS_PERC_MAX color208:color-1 VISITORS",
"COLOR_MTRC_HITS_PERC_MAX color208:color-1 OS",
"COLOR_MTRC_HITS_PERC_MAX color208:color-1 BROWSERS",
"COLOR_MTRC_HITS_PERC_MAX color208:color-1 VISIT_TIMES",
"COLOR_MTRC_VISITORS_PERC color187:color-1",
"COLOR_MTRC_VISITORS_PERC_MAX color208:color-1",
"COLOR_MTRC_BW_PERC color187:color-1",
"COLOR_MTRC_BW_PERC_MAX color208:color-1",
"COLOR_PANEL_COLS color242:color-1",
"COLOR_BARS color240:color-1",
"COLOR_SUBBARS color250:color-1",
"COLOR_CHART_AXIS color241:color-1",
"COLOR_CHART_VALUES color248:color-1",
"COLOR_ERROR color231:color197",
"COLOR_SELECTED color0:color215",
"COLOR_PANEL_ACTIVE color7:color240",
"COLOR_PANEL_HEADER color7:color237",
"COLOR_PANEL_DESC color242:color-1",
"COLOR_OVERALL_LBLS color251:color-1",
"COLOR_OVERALL_VALS color148:color-1",
"COLOR_OVERALL_PATH color186:color-1",
"COLOR_ACTIVE_LABEL color7:color237",
"COLOR_BG color7:color-1",
"COLOR_DEFAULT color7:color-1",
"COLOR_PROGRESS color7:color141",
};
static const char *const colors8_mono[] = {
"COLOR_MTRC_HITS color7:color-1",
"COLOR_MTRC_VISITORS color0:color-1 bold",
"COLOR_MTRC_DATA color7:color-1",
"COLOR_MTRC_BW color0:color-1 bold",
"COLOR_MTRC_AVGTS color0:color-1 bold",
"COLOR_MTRC_CUMTS color0:color-1 bold",
"COLOR_MTRC_MAXTS color0:color-1 bold",
"COLOR_MTRC_PROT color0:color-1 bold",
"COLOR_MTRC_MTHD color7:color-1 ",
"COLOR_MTRC_HITS_PERC color0:color-1 bold",
"COLOR_MTRC_HITS_PERC color1:color-1 bold VISITORS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold OS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold BROWSERS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold VISIT_TIMES",
"COLOR_MTRC_HITS_PERC_MAX color0:color-1 bold",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold VISITORS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold OS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold BROWSERS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold VISIT_TIMES",
"COLOR_MTRC_VISITORS_PERC color0:color-1 bold",
"COLOR_MTRC_VISITORS_PERC_MAX color0:color-1 bold",
"COLOR_MTRC_BW_PERC color0:color-1 bold",
"COLOR_MTRC_BW_PERC_MAX color0:color-1 bold",
"COLOR_PANEL_COLS color7:color-1",
"COLOR_BARS color7:color-1",
"COLOR_SUBBARS color8:color-1",
"COLOR_CHART_AXIS color7:color-1",
"COLOR_CHART_VALUES color7:color-1",
"COLOR_ERROR color7:color1",
"COLOR_SELECTED color0:color7",
"COLOR_PANEL_ACTIVE color0:color3",
"COLOR_PANEL_HEADER color0:color7",
"COLOR_PANEL_DESC color7:color-1",
"COLOR_OVERALL_LBLS color7:color-1 bold",
"COLOR_OVERALL_VALS color6:color-1",
"COLOR_OVERALL_PATH color3:color-1",
"COLOR_ACTIVE_LABEL color4:color7",
"COLOR_BG color7:color-1",
"COLOR_DEFAULT color7:color-1",
"COLOR_PROGRESS color0:color6",
};
static const char *const colors8_green[] = {
"COLOR_MTRC_HITS color7:color-1",
"COLOR_MTRC_VISITORS color0:color-1 bold",
"COLOR_MTRC_DATA color7:color-1",
"COLOR_MTRC_BW color0:color-1 bold",
"COLOR_MTRC_AVGTS color0:color-1 bold",
"COLOR_MTRC_CUMTS color0:color-1 bold",
"COLOR_MTRC_MAXTS color0:color-1 bold",
"COLOR_MTRC_PROT color0:color-1 bold",
"COLOR_MTRC_MTHD color7:color-1 ",
"COLOR_MTRC_HITS_PERC color0:color-1 bold",
"COLOR_MTRC_HITS_PERC color1:color-1 bold VISITORS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold OS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold BROWSERS",
"COLOR_MTRC_HITS_PERC color1:color-1 bold VISIT_TIMES",
"COLOR_MTRC_HITS_PERC_MAX color0:color-1 bold",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold VISITORS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold OS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold BROWSERS",
"COLOR_MTRC_HITS_PERC_MAX color3:color-1 bold VISIT_TIMES",
"COLOR_MTRC_VISITORS_PERC color0:color-1 bold",
"COLOR_MTRC_VISITORS_PERC_MAX color0:color-1 bold",
"COLOR_MTRC_BW_PERC color0:color-1 bold",
"COLOR_MTRC_BW_PERC_MAX color0:color-1 bold",
"COLOR_PANEL_COLS color7:color-1",
"COLOR_BARS color2:color-1",
"COLOR_SUBBARS color8:color-1",
"COLOR_CHART_AXIS color2:color-1",
"COLOR_CHART_VALUES color2:color-1",
"COLOR_ERROR color7:color1",
"COLOR_SELECTED color0:color7",
"COLOR_PANEL_ACTIVE color0:color3",
"COLOR_PANEL_HEADER color0:color2",
"COLOR_PANEL_DESC color7:color-1",
"COLOR_OVERALL_LBLS color7:color-1 bold",
"COLOR_OVERALL_VALS color6:color-1",
"COLOR_OVERALL_PATH color3:color-1",
"COLOR_ACTIVE_LABEL color0:color2",
"COLOR_BG color7:color-1",
"COLOR_DEFAULT color7:color-1",
"COLOR_PROGRESS color0:color6",
};
static const char *const nocolors[] = {
"COLOR_MTRC_HITS color0:color-1",
"COLOR_MTRC_VISITORS color0:color-1",
"COLOR_MTRC_DATA color0:color-1",
"COLOR_MTRC_BW color0:color-1",
"COLOR_MTRC_AVGTS color0:color-1",
"COLOR_MTRC_CUMTS color0:color-1",
"COLOR_MTRC_MAXTS color0:color-1",
"COLOR_MTRC_PROT color0:color-1",
"COLOR_MTRC_MTHD color0:color-1",
"COLOR_MTRC_HITS_PERC color0:color-1",
"COLOR_MTRC_HITS_PERC_MAX color0:color-1",
"COLOR_MTRC_VISITORS_PERC color0:color-1",
"COLOR_MTRC_VISITORS_PERC_MAX color0:color-1",
"COLOR_MTRC_BW_PERC color0:color-1",
"COLOR_MTRC_BW_PERC_MAX color0:color-1",
"COLOR_PANEL_COLS color0:color-1",
"COLOR_BARS color0:color-1",
"COLOR_SUBBARS color0:color-1",
"COLOR_CHART_AXIS color0:color-1",
"COLOR_CHART_VALUES color0:color-1",
"COLOR_ERROR color0:color-1",
"COLOR_SELECTED color0:color-1 reverse",
"COLOR_PANEL_ACTIVE color0:color-1 reverse",
"COLOR_PANEL_HEADER color0:color-1 reverse",
"COLOR_PANEL_DESC color0:color-1",
"COLOR_OVERALL_LBLS color0:color-1",
"COLOR_OVERALL_VALS color0:color-1",
"COLOR_OVERALL_PATH color0:color-1",
"COLOR_ACTIVE_LABEL color0:color-1 reverse",
"COLOR_BG color0:color-1",
"COLOR_DEFAULT color0:color-1",
"COLOR_PROGRESS color0:color-1 reverse",
};
/* *INDENT-ON* */
/* Allocate memory for color elements */
static GColors *
new_gcolors (void) {
GColors *color = xcalloc (1, sizeof (GColors));
color->module = -1;
return color;
}
/* Allocate memory for a color element properties */
static GColorPair *
new_gcolorpair (void) {
GColorPair *pair = xcalloc (1, sizeof (GColorPair));
/* Must be between 2 and COLOR_PAIRS-1.
* Starts at 2 since COLOR_NORMAL has already been set */
pair->idx = 2;
return pair;
}
/* Free malloc'd memory for color elements */
void
free_color_lists (void) {
if (pair_list)
list_remove_nodes (pair_list);
if (color_list)
list_remove_nodes (color_list);
color_list = NULL;
pair_list = NULL;
}
/* Set a default color - COLOR_NORMAL, this will be used if
* no colors are supported by the terminal */
void
set_normal_color (void) {
GColorPair *pair = new_gcolorpair ();
GColors *color = new_gcolors ();
pair->idx = 1;
pair->fg = COLOR_WHITE;
pair->bg = -1;
color->pair = pair;
color->item = COLOR_NORMAL;
pair_list = list_create (pair);
color_list = list_create (color);
init_pair (pair->idx, pair->fg, pair->bg);
}
/* Get color properties for COLOR_OVERALL_LBLS */
GColors *
color_overall_lbls (void) {
return get_color (COLOR_OVERALL_LBLS);
}
/* Get color properties for COLOR_OVERALL_VALS */
GColors *
color_overall_vals (void) {
return get_color (COLOR_OVERALL_VALS);
}
/* Get color properties for COLOR_OVERALL_PATH */
GColors *
color_overall_path (void) {
return get_color (COLOR_OVERALL_PATH);
}
/* Get color properties for COLOR_PANEL_HEADER */
GColors *
color_panel_header (void) {
return get_color (COLOR_PANEL_HEADER);
}
/* Get color properties for COLOR_PANEL_DESC */
GColors *
color_panel_desc (void) {
return get_color (COLOR_PANEL_DESC);
}
/* Get color properties for COLOR_PANEL_ACTIVE*/
GColors *
color_panel_active (void) {
return get_color (COLOR_PANEL_ACTIVE);
}
/* Get color properties for COLOR_SELECTED */
GColors *
color_selected (void) {
return get_color (COLOR_SELECTED);
}
/* Get color properties for COLOR_PROGRESS */
GColors *
color_progress (void) {
return get_color (COLOR_PROGRESS);
}
/* Get color properties for COLOR_DEFAULT */
GColors *
color_default (void) {
return get_color (COLOR_DEFAULT);
}
/* Get color properties for COLOR_ERROR */
GColors *
color_error (void) {
return get_color (COLOR_ERROR);
}
/* Get the enumerated color given its equivalent color string.
*
* On error, -1 is returned.
* On success, the enumerated color is returned. */
static int
get_color_item_enum (const char *str) {
return str2enum (CSTM_COLORS, ARRAY_SIZE (CSTM_COLORS), str);
}
/* Extract color number from the given config string.
*
* On error, -2 is returned. If color is greater than max colors, it aborts.
* On success, the color number is returned. */
static int
extract_color (char *color) {
char *sEnd;
int col = 0;
if (strncasecmp (color, "color", 5) != 0)
return -2;
color += 5;
col = strtol (color, &sEnd, 10);
if (color == sEnd || *sEnd != '\0' || errno == ERANGE)
return -2;
/* ensure used color is supported by the terminal */
if (col > COLORS)
FATAL ("Terminal doesn't support color: %d - max colors: %d", col, COLORS);
return col;
}
/* Assign the background and foreground color number from the given
* config string to GColorPair.
*
* On error, 1 is returned.
* On success, 0 is returned. */
static int
parse_bg_fg_color (GColorPair *pair, const char *value) {
char bgcolor[COLOR_STR_LEN] = "", fgcolor[COLOR_STR_LEN] = "";
int ret = 0;
if (sscanf (value, "%8[^:]:%8[^ ]", fgcolor, bgcolor) != 2)
return 1;
if ((pair->bg = extract_color (bgcolor)) == -2)
ret = 1;
if ((pair->fg = extract_color (fgcolor)) == -2)
ret = 1;
return ret;
}
/* Assign color attributes from the given config string to GColors. */
static void
locate_attr_color (GColors *color, const char *attr) {
if (strstr (attr, "bold"))
color->attr |= A_BOLD;
if (strstr (attr, "underline"))
color->attr |= A_UNDERLINE;
if (strstr (attr, "normal"))
color->attr |= A_NORMAL;
if (strstr (attr, "reverse"))
color->attr |= A_REVERSE;
if (strstr (attr, "standout"))
color->attr |= A_REVERSE;
if (strstr (attr, "blink"))
color->attr |= A_BLINK;
}
/* Parse color attributes from the given config string.
*
* On error, 1 is returned.
* On success, 0 is returned. */
static int
parse_attr_color (GColors *color, const char *value) {
char *line, *ptr, *start;
int ret = 0;
line = xstrdup (value);
start = strchr (line, ' ');
if ((!start) || (!*(start + 1))) {
LOG_DEBUG (("attempted to parse color attr: %s\n", value));
goto clean;
}
start++;
while (1) {
if ((ptr = strpbrk (start, ", ")) != NULL)
*ptr = 0;
locate_attr_color (color, start);
if (ptr == NULL)
break;
start = ptr + 1;
}
clean:
free (line);
return ret;
}
/* Parse color module from the given config string.
*
* On error, 1 is returned.
* On success, 0 is returned. */
static int
parse_module_color (GColors *color, const char *value) {
char *line = xstrdup (value), *p;
p = strrchr (line, ' ');
if (!p || !*(p + 1)) {
LOG_DEBUG (("attempted to parse color module: %s\n", value));
goto clean;
}
if ((color->module = get_module_enum (p + 1)) == -1)
LOG_DEBUG (("attempted to parse color module: %s\n", value));
clean:
free (line);
return 0;
}
/* Find a color by item and module attributes on the list of already
* parsed colors.
*
* If color exists, 1 is returned.
* If color does not exist, 1 is returned. */
static int
find_color_in_list (void *data, void *color) {
GColors *new_color = color;
GColors *old_color = data;
if (old_color->item != new_color->item)
return 0;
if (old_color->module != new_color->module)
return 0;
return 1;
}
/* Find a color by foreground and background attributes on the list of
* already parsed colors.
*
* If color exists, 1 is returned.
* If color does not exist, 1 is returned. */
static int
find_pair_in_list (void *data, void *color) {
GColorPair *new_color = color;
GColorPair *old_color = data;
if (old_color->fg != new_color->fg)
return 0;
if (old_color->bg != new_color->bg)
return 0;
return 1;
}
/* Compare a color item (GColorItem) that has no module with the given needle
* item.
*
* If the items match and with no module, 1 is returned.
* If condition is not satisfied, 0 is returned. */
static int
find_color_item_in_list (void *data, void *needle) {
GColors *color = data;
GColorItem *item = needle;
return color->item == (GColorItem) (*(int *) item) && color->module == -1;
}
/* Compare a color item (GColorItem) and module with the given needle item.
*
* If the items match and with no module, 1 is returned.
* If condition is not satisfied, 0 is returned. */
static int
find_color_item_module_in_list (void *data, void *needle) {
GColors *color = data;
GColors *item = needle;
return color->item == item->item && color->module == item->module;
}
/* Get color item properties given an item (enumerated).
*
* On error, it aborts.
* On success, the color item properties are returned, or NULL if no match
* found. */
GColors *
get_color (GColorItem item) {
GColorItem normal = COLOR_NORMAL;
GSLList *match = NULL;
if ((match = list_find (color_list, find_color_item_in_list, &item)))
return (GColors *) match->data;
if ((match = list_find (color_list, find_color_item_in_list, &normal)))
return (GColors *) match->data;
/* should not get here */
FATAL ("Unable to find color item %d", item);
}
/* Get color item properties given an item (enumerated) and its module.
*
* On error, it aborts.
* On success, the color item properties are returned, or NULL if no match
* found. */
GColors *
get_color_by_item_module (GColorItem item, GModule module) {
GColors *needle = new_gcolors (), *color = NULL;
GSLList *match = NULL;
needle->module = module;
needle->item = item;
/* find color for specific item/module pair */
if ((match = list_find (color_list, find_color_item_module_in_list, needle)))
color = match->data;
/* attempt to find color by item (fallback) */
if (!color)
color = get_color (item);
free (needle);
return color;
}
/* Parse a color definition line from the config file.
*
* On error, it aborts.
* On success, the color properties are assigned */
static void
parse_color_line (GColorPair *pair, GColors *color, char *line) {
char *val;
int item = 0;
size_t idx;
/* key */
idx = strcspn (line, " \t");
if (strlen (line) == idx)
FATAL ("Malformed color key at line: %s", line);
line[idx] = '\0';
if ((item = get_color_item_enum (line)) == -1)
FATAL ("Unable to find color key: %s", line);
/* value */
val = line + (idx + 1);
idx = strspn (val, " \t");
if (strlen (val) == idx)
FATAL ("Malformed color value at line: %s", line);
val = val + idx;
/* get background/foreground color */
if (parse_bg_fg_color (pair, val) == 1)
FATAL ("Invalid bg/fg color pairs at: %s %s", line, val);
if (parse_attr_color (color, val) == 1)
FATAL ("Invalid color attrs at: %s %s", line, val);
if (parse_module_color (color, val) == 1)
FATAL ("Invalid color module at: %s %s", line, val);
color->item = item;
}
/* Attempt to prepend the given color on our color linked list.
*
* On error, or if color already exists, the given color is freed.
* On success, or if not color found, store color properties */
static void
prepend_color (GColors **color) {
/* create a list of colors if one does not exist */
if (color_list == NULL) {
color_list = list_create (*color);
}
/* attempt to find the given color data type (by item and attributes) in
* our color list */
else if (list_find (color_list, find_color_in_list, *color)) {
/* if found, free the recently malloc'd color data type and use
* existing color */
free (*color);
*color = NULL;
} else {
/* not a dup, so insert the new color in our color list */
color_list = list_insert_prepend (color_list, *color);
}
}
/* Parse a color definition line from the config file and store it on a single
* linked-list.
*
* On error, it aborts.
* On success, the color properties are stored */
static void
parse_color (char *line) {
GSLList *match = NULL;
GColors *color = NULL;
GColorPair *pair = NULL;
color = new_gcolors ();
pair = new_gcolorpair ();
/* extract a color pair and color attributes from the given config line */
parse_color_line (pair, color, line);
/* create a pair color list if one doesn't exist */
if (pair_list == NULL) {
pair_list = list_create (pair);
}
/* attempt to find the given color pair in our pair list */
else if ((match = list_find (pair_list, find_pair_in_list, pair))) {
/* pair found, use new pair and free existing one */
free (pair);
pair = (GColorPair *) match->data;
}
/* pair not found, use it then */
else {
pair->idx += list_count (pair_list);
pair_list = list_insert_prepend (pair_list, pair);
}
/* set color pair */
color->pair = pair;
prepend_color (&color);
/* if no color pair was found, then we init the color pair */
if (!match && color)
init_pair (color->pair->idx, color->pair->fg, color->pair->bg);
free (line);
}
/* Iterate over all color definitions in the config file.
*
* On error, it aborts.
* On success, the color properties are parsed and stored */
static void
parse_colors (const char *const colors[], size_t n) {
char *line;
size_t i;
for (i = 0; i < n; ++i) {
line = strdup (colors[i]);
/* did not find a valid format */
if (strchr (line, ':') == NULL) {
free (line);
continue;
}
parse_color (line);
}
}
/* Use default color definitions if necessary. */
static void
add_default_colors (void) {
/* no colors */
if (COLORS < 8)
parse_colors (nocolors, ARRAY_SIZE (nocolors));
/* 256 colors, and no color scheme set or set to monokai */
if (COLORS == 256 && (!conf.color_scheme || conf.color_scheme == MONOKAI))
parse_colors (colors256_monokai, ARRAY_SIZE (colors256_monokai));
/* otherwise use 16 colors scheme */
else if (COLORS > 16) {
if (conf.color_scheme == STD_GREEN)
parse_colors (colors256_green, ARRAY_SIZE (colors256_green));
else
parse_colors (colors256_mono, ARRAY_SIZE (colors256_mono));
}
/* 8 colors */
if (COLORS >= 8 && COLORS <= 16) {
if (conf.color_scheme == STD_GREEN)
parse_colors (colors8_green, ARRAY_SIZE (colors8_green));
else
parse_colors (colors8_mono, ARRAY_SIZE (colors8_mono));
}
}
/* Entry point to parse color definitions or use default colors */
void
set_colors (int force) {
errno = 0;
if (conf.color_idx > 0 && !force)
parse_colors (conf.colors, conf.color_idx);
else
add_default_colors ();
}
src/color.h
+115
View File
@@ -0,0 +1,115 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef COLOR_H_INCLUDED
#define COLOR_H_INCLUDED
#define COLOR_STR_LEN 9
/* Color Items/Fields */
typedef enum CSTM_COLORS {
COLOR_NORMAL,
COLOR_MTRC_HITS,
COLOR_MTRC_VISITORS,
COLOR_MTRC_DATA,
COLOR_MTRC_BW,
COLOR_MTRC_AVGTS,
COLOR_MTRC_CUMTS,
COLOR_MTRC_MAXTS,
COLOR_MTRC_PROT,
COLOR_MTRC_MTHD,
COLOR_MTRC_HITS_PERC,
COLOR_MTRC_HITS_PERC_MAX,
COLOR_MTRC_VISITORS_PERC,
COLOR_MTRC_VISITORS_PERC_MAX,
COLOR_MTRC_BW_PERC,
COLOR_MTRC_BW_PERC_MAX,
COLOR_PANEL_COLS,
COLOR_BARS,
COLOR_SUBBARS,
COLOR_CHART_AXIS,
COLOR_CHART_VALUES,
COLOR_ERROR,
COLOR_SELECTED,
COLOR_PANEL_ACTIVE,
COLOR_PANEL_HEADER,
COLOR_PANEL_DESC,
COLOR_OVERALL_LBLS,
COLOR_OVERALL_VALS,
COLOR_OVERALL_PATH,
COLOR_ACTIVE_LABEL,
COLOR_BG,
COLOR_DEFAULT,
COLOR_PROGRESS,
} GColorItem;
/* Default Color Schemes */
typedef enum SCHEMES {
NO_COLOR,
MONOCHROME,
STD_GREEN,
MONOKAI,
} GSchemes;
#include "commons.h"
/* Each color properties */
typedef struct GColorPair_ {
short idx; /* color pair index identifier */
short fg; /* foreground color */
short bg; /* background color */
} GColorPair;
/* Color */
typedef struct GColors_ {
GColorItem item; /* screen item */
GColorPair *pair; /* color pair */
int attr; /* color attributes, e.g., bold */
short module; /* panel */
int pair_idx;
} GColors;
GColors *color_default (void);
GColors *color_error (void);
GColors *color_overall_lbls (void);
GColors *color_overall_path (void);
GColors *color_overall_vals (void);
GColors *color_panel_active (void);
GColors *color_panel_desc (void);
GColors *color_panel_header (void);
GColors *color_progress (void);
GColors *color_selected (void);
GColors *get_color_by_item_module (GColorItem item, GModule module);
GColors *get_color (GColorItem item);
GColors *get_color_normal (void);
void free_color_lists (void);
void set_colors (int force);
void set_normal_color (void);
#endif // for #ifndef COLOR_H
src/commons.c
+521
View File
@@ -0,0 +1,521 @@
/**
* commons.c -- holds different data types
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include "commons.h"
#include "labels.h"
#include "settings.h"
#include "util.h"
#include "xmalloc.h"
/* processing time */
time_t end_proc;
time_t timestamp;
time_t start_proc;
/* list of available modules/panels */
int module_list[TOTAL_MODULES] = {[0 ... TOTAL_MODULES - 1] = -1 };
/* *INDENT-OFF* */
/* String modules to enumerated modules */
static const GEnum enum_modules[] = {
{"VISITORS" , VISITORS} ,
{"REQUESTS" , REQUESTS} ,
{"REQUESTS_STATIC" , REQUESTS_STATIC} ,
{"NOT_FOUND" , NOT_FOUND} ,
{"HOSTS" , HOSTS} ,
{"OS" , OS} ,
{"BROWSERS" , BROWSERS} ,
{"VISIT_TIMES" , VISIT_TIMES} ,
{"VIRTUAL_HOSTS" , VIRTUAL_HOSTS} ,
{"REFERRERS" , REFERRERS} ,
{"REFERRING_SITES" , REFERRING_SITES} ,
{"KEYPHRASES" , KEYPHRASES} ,
{"STATUS_CODES" , STATUS_CODES} ,
{"REMOTE_USER" , REMOTE_USER} ,
{"CACHE_STATUS" , CACHE_STATUS} ,
#ifdef HAVE_GEOLOCATION
{"GEO_LOCATION" , GEO_LOCATION} ,
{"ASN" , ASN} ,
#endif
{"MIME_TYPE" , MIME_TYPE} ,
{"TLS_TYPE" , TLS_TYPE} ,
};
/* *INDENT-ON* */
/* Get number of items per panel/level to parse.
*
* If is_sub is true, returns sub-item limits, otherwise returns panel limits.
* The number of items per panel/level is returned. */
static uint32_t
get_max_choices_internal (int is_sub) {
char *csv = NULL, *json = NULL, *html = NULL;
int max = MAX_CHOICES;
int rt_default = is_sub ? MAX_CHOICES_SUB_RT : MAX_CHOICES_RT;
/* no max choices, return defaults */
if (conf.max_items <= 0)
return conf.real_time_html ? rt_default : (conf.date_spec_hr == 2 ? MAX_CHOICES_MINUTE : MAX_CHOICES);
/* TERM */
if (!conf.output_stdout)
return conf.max_items > MAX_CHOICES ? MAX_CHOICES : conf.max_items;
/* REAL-TIME STDOUT */
/* real time HTML, display max rt choices - always cap at MAX_CHOICES_RT */
if (conf.real_time_html)
return conf.max_items > MAX_CHOICES_RT ? MAX_CHOICES_RT : conf.max_items;
/* STDOUT */
/* CSV - allow n amount of choices */
if (find_output_type (&csv, "csv", 1) == 0)
max = conf.max_items;
/* JSON - allow n amount of choices */
if (find_output_type (&json, "json", 1) == 0 && conf.max_items > 0)
max = conf.max_items;
/* HTML - takes priority on cases where multiple outputs were given. Note that
* we check either for an .html extension or we assume not extension was passed
* via -o and therefore we are redirecting the output to a file. */
if (find_output_type (&html, "html", 1) == 0 || conf.output_format_idx == 0)
max = conf.max_items;
free (csv);
free (html);
free (json);
return max;
}
/* Get number of items per panel to parse. */
uint32_t
get_max_choices (void) {
return get_max_choices_internal (0);
}
/* Get number of sub-items per hierarchy level to parse. */
uint32_t
get_max_choices_sub (void) {
return get_max_choices_internal (1);
}
/* Calculate a percentage.
*
* The percentage is returned. */
float
get_percentage (unsigned long long total, unsigned long long hit) {
return (total == 0 ? 0 : (((float) hit) / total) * 100);
}
/* Display the storage being used. */
void
display_storage (void) {
fprintf (stdout, "%s\n", BUILT_WITH_DEFHASH);
}
/* Display the path of the default configuration file when `-p` is not used */
void
display_default_config_file (void) {
char *path = get_config_file_path ();
if (!path) {
fprintf (stdout, "%s\n", ERR_NODEF_CONF_FILE);
fprintf (stdout, "%s `-p /path/goaccess.conf`\n", ERR_NODEF_CONF_FILE_DESC);
} else {
fprintf (stdout, "%s\n", path);
free (path);
}
}
/* Display the current version. */
void
display_version (void) {
fprintf (stdout, "GoAccess - %s.\n", GO_VERSION);
fprintf (stdout, "%s: %s\n", INFO_MORE_INFO, GO_WEBSITE);
fprintf (stdout, "Copyright (C) 2009-2024 by Gerardo Orellana\n");
fprintf (stdout, "\nBuild configure arguments:\n");
#ifdef DEBUG
fprintf (stdout, " --enable-debug\n");
#endif
#ifdef HAVE_NCURSESW_NCURSES_H
fprintf (stdout, " --enable-utf8\n");
#endif
#ifdef HAVE_LIBGEOIP
fprintf (stdout, " --enable-geoip=legacy\n");
#endif
#ifdef HAVE_LIBMAXMINDDB
fprintf (stdout, " --enable-geoip=mmdb\n");
#endif
#ifdef WITH_GETLINE
fprintf (stdout, " --with-getline\n");
#endif
#ifdef HAVE_LIBSSL
fprintf (stdout, " --with-openssl\n");
#endif
}
/* Get the enumerated value given a string.
*
* On error, -1 is returned.
* On success, the enumerated module value is returned. */
int
str2enum (const GEnum map[], int len, const char *str) {
int i;
for (i = 0; i < len; ++i) {
if (!strcmp (str, map[i].str))
return map[i].idx;
}
return -1;
}
/* Get the string value given an enum.
*
* On error, -1 is returned.
* On success, the enumerated module value is returned. */
const char *
enum2str (const GEnum map[], int len, int idx) {
int i;
for (i = 0; i < len; ++i) {
if (idx == map[i].idx)
return map[i].str;
}
return NULL;
}
/* Get the enumerated module value given a module string.
*
* On error, -1 is returned.
* On success, the enumerated module value is returned. */
int
get_module_enum (const char *str) {
return str2enum (enum_modules, ARRAY_SIZE (enum_modules), str);
}
/* Get the module string value given a module enum value.
*
* On error, NULL is returned.
* On success, the string module value is returned. */
const char *
get_module_str (GModule module) {
return enum2str (enum_modules, ARRAY_SIZE (enum_modules), module);
}
/* Instantiate a new GAgents structure.
*
* On success, the newly malloc'd structure is returned. */
GAgents *
new_gagents (uint32_t size) {
GAgents *agents = xmalloc (sizeof (GAgents));
memset (agents, 0, sizeof *agents);
agents->items = xcalloc (size, sizeof (GAgentItem));
agents->size = size;
agents->idx = 0;
return agents;
}
/* Clean the array of agents. */
void
free_agents_array (GAgents *agents) {
int i;
if (agents == NULL)
return;
/* clean stuff up */
for (i = 0; i < agents->idx; ++i)
free (agents->items[i].agent);
if (agents->items)
free (agents->items);
free (agents);
}
/* Determine if the given date format is a timestamp.
*
* If not a timestamp, 0 is returned.
* If it is a timestamp, 1 is returned. */
int
has_timestamp (const char *fmt) {
if (strcmp ("%s", fmt) == 0 || strcmp ("%f", fmt) == 0)
return 1;
return 0;
}
/* Determine if the given module is set to be enabled.
*
* If enabled, 1 is returned, else 0 is returned. */
int
enable_panel (GModule mod) {
int i, module;
for (i = 0; i < conf.enable_panel_idx; ++i) {
if ((module = get_module_enum (conf.enable_panels[i])) == -1)
continue;
if (mod == (unsigned int) module) {
return 1;
}
}
return 0;
}
/* Determine if the given module is set to be ignored.
*
* If ignored, 1 is returned, else 0 is returned. */
int
ignore_panel (GModule mod) {
int i, module;
for (i = 0; i < conf.ignore_panel_idx; ++i) {
if ((module = get_module_enum (conf.ignore_panels[i])) == -1)
continue;
if (mod == (unsigned int) module) {
return 1;
}
}
return 0;
}
/* Get the number of available modules/panels.
*
* The number of modules available is returned. */
uint32_t
get_num_modules (void) {
size_t idx = 0;
uint32_t num = 0;
FOREACH_MODULE (idx, module_list) {
num++;
}
return num;
}
/* Get the index from the module_list given a module.
*
* If the module is not within the array, -1 is returned.
* If the module is within the array, the index is returned. */
int
get_module_index (int module) {
size_t idx = 0;
FOREACH_MODULE (idx, module_list) {
if (module_list[idx] == module)
return idx;
}
return -1;
}
/* Remove the given module from the module_list array.
*
* If the module is not within the array, 1 is returned.
* If the module is within the array, it is removed from the array and
* 0 is returned. */
int
remove_module (GModule module) {
int idx = get_module_index (module);
if (idx == -1)
return 1;
if (idx < TOTAL_MODULES - 1)
memmove (&module_list[idx], &module_list[idx + 1],
((TOTAL_MODULES - 1) - idx) * sizeof (module_list[0]));
module_list[TOTAL_MODULES - 1] = -1;
return 0;
}
/* Find the next module given the current module.
*
* The next available module in the array is returned. */
int
get_next_module (GModule module) {
int next = get_module_index (module) + 1;
if (next == TOTAL_MODULES || module_list[next] == -1)
return module_list[0];
return module_list[next];
}
/* Find the previous module given the current module.
*
* The previous available module in the array is returned. */
int
get_prev_module (GModule module) {
int i;
int next = get_module_index (module) - 1;
if (next >= 0 && module_list[next] != -1)
return module_list[next];
for (i = TOTAL_MODULES - 1; i >= 0; i--) {
if (module_list[i] != -1) {
return module_list[i];
}
}
return 0;
}
/* Perform some additional tasks to panels before they are being
* parsed.
*
* Note: This overwrites --enable-panel since it assumes there's
* truly nothing to do with the panel */
void
verify_panels (void) {
int ignore_panel_idx = conf.ignore_panel_idx;
/* Remove virtual host panel if no '%v' within log format */
if (!conf.log_format)
return;
if (!strstr (conf.log_format, "%v") && ignore_panel_idx < TOTAL_MODULES && !conf.fname_as_vhost) {
if (str_inarray ("VIRTUAL_HOSTS", conf.ignore_panels, ignore_panel_idx) < 0)
remove_module (VIRTUAL_HOSTS);
}
if (!strstr (conf.log_format, "%e") && ignore_panel_idx < TOTAL_MODULES) {
if (str_inarray ("REMOTE_USER", conf.ignore_panels, ignore_panel_idx) < 0)
remove_module (REMOTE_USER);
}
if (!strstr (conf.log_format, "%C") && ignore_panel_idx < TOTAL_MODULES) {
if (str_inarray ("CACHE_STATUS", conf.ignore_panels, ignore_panel_idx) < 0)
remove_module (CACHE_STATUS);
}
if (!strstr (conf.log_format, "%M") && ignore_panel_idx < TOTAL_MODULES) {
if (str_inarray ("MIME_TYPE", conf.ignore_panels, ignore_panel_idx) < 0)
remove_module (MIME_TYPE);
}
if (!strstr (conf.log_format, "%K") && ignore_panel_idx < TOTAL_MODULES) {
if (str_inarray ("TLS_TYPE", conf.ignore_panels, ignore_panel_idx) < 0)
remove_module (TLS_TYPE);
}
#ifdef HAVE_GEOLOCATION
#ifdef HAVE_LIBMAXMINDDB
if (!conf.geoip_db_idx && ignore_panel_idx < TOTAL_MODULES) {
if (str_inarray ("GEO_LOCATION", conf.ignore_panels, ignore_panel_idx) < 0)
remove_module (GEO_LOCATION);
}
if (!conf.geoip_db_idx && ignore_panel_idx < TOTAL_MODULES) {
if (str_inarray ("ASN", conf.ignore_panels, ignore_panel_idx) < 0)
remove_module (ASN);
}
#endif
#endif
}
/* Build an array of available modules (ignores listed panels).
*
* If there are no modules enabled, 0 is returned.
* On success, the first enabled module is returned. */
int
init_modules (void) {
GModule module;
int i;
/* init - terminating with -1 */
for (module = 0; module < TOTAL_MODULES; ++module)
module_list[module] = -1;
for (i = 0, module = 0; module < TOTAL_MODULES; ++module) {
if (!ignore_panel (module) || enable_panel (module)) {
module_list[i++] = module;
}
}
return module_list[0] > -1 ? module_list[0] : 0;
}
/* Get the logs size.
*
* If log was piped (from stdin), 0 is returned.
* On success, it adds up all log sizes and its value is returned.
* if --log-size was specified, it will be returned explicitly */
intmax_t
get_log_sizes (void) {
int i;
off_t size = 0;
/* --log-size */
if (conf.log_size > 0)
return (intmax_t) conf.log_size;
for (i = 0; i < conf.filenames_idx; ++i) {
if (conf.filenames[i][0] == '-' && conf.filenames[i][1] == '\0')
size += 0;
else
size += file_size (conf.filenames[i]);
}
return (intmax_t) size;
}
/* Get the log sources used.
*
* On success, a newly malloc'd string containing the log source either
* from filename(s) and/or STDIN is returned. */
char *
get_log_source_str (int max_len) {
char *str = xstrdup ("");
int i, len = 0;
for (i = 0; i < conf.filenames_idx; ++i) {
if (conf.filenames[i][0] == '-' && conf.filenames[i][1] == '\0')
append_str (&str, "STDIN");
else
append_str (&str, conf.filenames[i]);
if (i != conf.filenames_idx - 1)
append_str (&str, "; ");
}
len = strlen (str);
if (max_len > 0 && len > 0 && len > max_len) {
str[max_len - 3] = 0;
append_str (&str, "...");
}
return str;
}
src/commons.h
+296
View File
@@ -0,0 +1,296 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#ifndef COMMONS_H_INCLUDED
#define COMMONS_H_INCLUDED
#include <time.h>
#include <stdint.h>
#include "gslist.h"
/* Remove the __attribute__ stuff when the compiler is not GCC. */
#if !__GNUC__
#define __attribute__(x) /**/
#endif
#define GO_UNUSED __attribute__((unused))
#define GO_VERSION "1.10.2"
#define GO_WEBSITE "https://goaccess.io/"
extern struct tm now_tm;
/* common char array buffer size */
#define INIT_BUF_SIZE 1024
/* total number of modules */
#ifdef HAVE_GEOLOCATION
#define TOTAL_MODULES 19
#else
#define TOTAL_MODULES 17
#endif
/* maximum number of items within a panel */
#define MAX_CHOICES 366
/* real-time */
#define MAX_CHOICES_RT 50
/* real-time sub-items */
#define MAX_CHOICES_SUB_RT 10
/* max default items when date-spec = min */
#define MAX_CHOICES_MINUTE 1440 /* 24hrs */
/* date and time length - e.g., 2016/12/12 12:12:12 -0600 */
#define DATE_TIME 25 + 1
/* date length - e.g., 2016/12/12 */
#define DATE_LEN 10 + 1
/* date length - e.g., 12:12:12 */
#define TIME_LEN 8 + 1
/* hour + ':' + min length - e.g., 12:12 */
#define HRMI_LEN 4 + 1 + 1
#define YR_FMT "%Y"
#define MO_FMT "%M"
#define DT_FMT "%d"
/* maximum protocol string length */
#define REQ_PROTO_LEN 9
#define IGNORE_LEVEL_PANEL 1
#define IGNORE_LEVEL_REQ 2
/* Type of IP */
typedef enum {
TYPE_IPINV,
TYPE_IPV4,
TYPE_IPV6
} GTypeIP;
/* Type of Modules */
typedef enum MODULES {
VISITORS,
REQUESTS,
REQUESTS_STATIC,
NOT_FOUND,
HOSTS,
OS,
BROWSERS,
VISIT_TIMES,
VIRTUAL_HOSTS,
REFERRERS,
REFERRING_SITES,
KEYPHRASES,
STATUS_CODES,
REMOTE_USER,
CACHE_STATUS,
#ifdef HAVE_GEOLOCATION
GEO_LOCATION,
ASN,
#endif
MIME_TYPE,
TLS_TYPE,
} GModule;
/* Total number of storage metrics (GSMetric) */
#define GSMTRC_TOTAL 19
/* Enumerated Storage Metrics */
typedef enum GSMetric_ {
MTRC_KEYMAP,
MTRC_ROOTMAP,
MTRC_DATAMAP,
MTRC_UNIQMAP,
MTRC_ROOT,
MTRC_HITS,
MTRC_VISITORS,
MTRC_BW,
MTRC_CUMTS,
MTRC_MAXTS,
MTRC_METHODS,
MTRC_PROTOCOLS,
MTRC_AGENTS,
MTRC_METADATA,
MTRC_UNIQUE_KEYS,
MTRC_AGENT_KEYS,
MTRC_AGENT_VALS,
MTRC_CNT_VALID,
MTRC_CNT_BW,
} GSMetric;
/* Metric totals. These are metrics that have a percent value and are
* calculated values. */
typedef struct GPercTotals_ {
uint32_t hits; /* total valid hits */
uint32_t visitors; /* total visitors */
uint64_t bw; /* total bandwidth */
} GPercTotals;
/* Metrics within GHolder or GDashData */
typedef struct GMetrics {
/* metric id can be used to identify
* a specific data field */
uint8_t id;
char *data;
char *method;
char *protocol;
float hits_perc;
float visitors_perc;
float bw_perc;
uint64_t hits;
uint64_t visitors;
uint64_t nbw;
double lat;
double lng;
/* holder has a numeric value, while
* dashboard has a displayable string value */
union {
char *sts;
uint64_t nts;
} avgts;
/* holder has a numeric value, while
* dashboard has a displayable string value */
union {
char *sts;
uint64_t nts;
} cumts;
/* holder has a numeric value, while
* dashboard has a displayable string value */
union {
char *sts;
uint64_t nts;
} maxts;
} GMetrics;
/* Forward declaration for recursive sub-items */
typedef struct GSubList_ GSubList;
/* Holder sub item */
typedef struct GSubItem_ {
GModule module;
GMetrics *metrics;
GSubList *sub_list;
struct GSubItem_ *prev;
struct GSubItem_ *next;
} GSubItem;
/* Double linked-list of sub items */
struct GSubList_ {
uint32_t size;
struct GSubItem_ *head;
struct GSubItem_ *tail;
};
/* Holder item */
typedef struct GHolderItem_ {
GSubList *sub_list;
GMetrics *metrics;
} GHolderItem;
/* Holder of GRawData */
typedef struct GHolder_ {
GHolderItem *items; /* holder items */
GModule module; /* current module */
uint32_t idx; /* holder index */
uint32_t holder_size; /* number of allocated items */
uint32_t ht_size; /* size of the hash table/store */
uint32_t sub_items_size; /* number of sub items */
uint32_t max_choices; /* max items at root level */
uint32_t max_choices_sub; /* max items at sub-item levels */
} GHolder;
/* Enum-to-string */
typedef struct GEnum_ {
const char *str;
int idx;
} GEnum;
/* A metric can contain a root/data/uniq node id */
typedef struct GDataMap_ {
int data;
int root;
} GDataMap;
typedef struct GAgentItem_ {
char *agent;
} GAgentItem;
typedef struct GAgents_ {
int size;
int idx;
struct GAgentItem_ *items;
} GAgents;
#define FOREACH_MODULE(item, array) \
for (; (item < ARRAY_SIZE(array)) && array[item] != -1; ++item)
/* Processing time */
extern time_t end_proc;
extern time_t timestamp;
extern time_t start_proc;
/* list of available modules/panels */
extern int module_list[TOTAL_MODULES];
/* *INDENT-OFF* */
GAgents *new_gagents (uint32_t size);
void free_agents_array (GAgents *agents);
const char *enum2str (const GEnum map[], int len, int idx);
const char *get_module_str (GModule module);
float get_percentage (unsigned long long total, unsigned long long hit);
uint32_t get_max_choices (void);
uint32_t get_max_choices_sub (void);
int get_module_enum (const char *str);
int has_timestamp (const char *fmt);
int str2enum (const GEnum map[], int len, const char *str);
int enable_panel (GModule mod);
int get_module_index (int module);
int get_next_module (GModule module);
int get_prev_module (GModule module);
int ignore_panel (GModule mod);
int init_modules (void);
int remove_module(GModule module);
uint32_t get_num_modules (void);
void verify_panels (void);
char *get_log_source_str (int max_len);
intmax_t get_log_sizes (void);
void display_default_config_file (void);
void display_storage (void);
void display_version (void);
/* *INDENT-ON* */
#endif
src/csv.c
+326
View File
@@ -0,0 +1,326 @@
/**
* output.c -- output csv to the standard output stream
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#define _LARGEFILE_SOURCE
#define _LARGEFILE64_SOURCE
#define _FILE_OFFSET_BITS 64
#include <ctype.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <inttypes.h>
#include "csv.h"
#include "error.h"
#include "gkhash.h"
#include "ui.h"
#include "util.h"
struct tm now_tm;
/* Panel output */
typedef struct GPanel_ {
GModule module;
void (*render) (FILE * fp, GHolder * h, GPercTotals totals);
} GPanel;
static void print_csv_data (FILE * fp, GHolder * h, GPercTotals totals);
/* *INDENT-OFF* */
/* A function pointer for each panel */
static const GPanel paneling[] = {
{VISITORS , print_csv_data} ,
{REQUESTS , print_csv_data} ,
{REQUESTS_STATIC , print_csv_data} ,
{NOT_FOUND , print_csv_data} ,
{HOSTS , print_csv_data} ,
{OS , print_csv_data} ,
{BROWSERS , print_csv_data} ,
{VISIT_TIMES , print_csv_data} ,
{VIRTUAL_HOSTS , print_csv_data} ,
{REFERRERS , print_csv_data} ,
{REFERRING_SITES , print_csv_data} ,
{KEYPHRASES , print_csv_data} ,
{STATUS_CODES , print_csv_data} ,
{REMOTE_USER , print_csv_data} ,
{CACHE_STATUS , print_csv_data} ,
#ifdef HAVE_GEOLOCATION
{GEO_LOCATION , print_csv_data} ,
{ASN , print_csv_data} ,
#endif
{MIME_TYPE , print_csv_data} ,
{TLS_TYPE , print_csv_data} ,
};
/* *INDENT-ON* */
/* Get a panel from the GPanel structure given a module.
*
* On error, or if not found, NULL is returned.
* On success, the panel value is returned. */
static const GPanel *
panel_lookup (GModule module) {
int i, num_panels = ARRAY_SIZE (paneling);
for (i = 0; i < num_panels; i++) {
if (paneling[i].module == module)
return &paneling[i];
}
return NULL;
}
/* Iterate over the string and escape CSV output. */
static void
escape_cvs_output (FILE *fp, char *s) {
while (*s) {
switch (*s) {
case '"':
fprintf (fp, "\"\"");
break;
default:
fputc (*s, fp);
break;
}
s++;
}
}
/* Output metrics.
*
* On success, outputs item value. */
static void
print_csv_metric_block (FILE *fp, GMetrics *nmetrics) {
/* basic metrics */
fprintf (fp, "\"%" PRIu64 "\",", nmetrics->hits);
fprintf (fp, "\"%4.2f%%\",", nmetrics->hits_perc);
fprintf (fp, "\"%" PRIu64 "\",", nmetrics->visitors);
fprintf (fp, "\"%4.2f%%\",", nmetrics->visitors_perc);
/* bandwidth */
if (conf.bandwidth) {
fprintf (fp, "\"%" PRIu64 "\",", nmetrics->nbw);
fprintf (fp, "\"%4.2f%%\",", nmetrics->bw_perc);
}
/* time served metrics */
if (conf.serve_usecs) {
fprintf (fp, "\"%" PRIu64 "\",", nmetrics->avgts.nts);
fprintf (fp, "\"%" PRIu64 "\",", nmetrics->cumts.nts);
fprintf (fp, "\"%" PRIu64 "\",", nmetrics->maxts.nts);
}
/* request method */
if (conf.append_method && nmetrics->method)
fprintf (fp, "\"%s\"", nmetrics->method);
fprintf (fp, ",");
/* request protocol */
if (conf.append_protocol && nmetrics->protocol)
fprintf (fp, "\"%s\"", nmetrics->protocol);
fprintf (fp, ",");
/* data field */
fprintf (fp, "\"");
escape_cvs_output (fp, nmetrics->data);
fprintf (fp, "\"\r\n");
}
/* Output a sublist (double linked-list) items for a particular parent node.
*
* On error, it exits early.
* On success, outputs item value. */
static void
print_csv_sub_items (FILE *fp, GSubList *sub_list, GModule module, int parent_idx,
GPercTotals totals, int depth) {
GMetrics *nmetrics;
GSubItem *iter;
uint32_t i = 0;
if (sub_list == NULL)
return;
for (iter = sub_list->head; iter; iter = iter->next, i++) {
set_data_metrics (iter->metrics, &nmetrics, totals);
fprintf (fp, "\"%u\",", i); /* idx */
fprintf (fp, "\"%d\",", parent_idx); /* parent idx */
fprintf (fp, "\"%s\",", module_to_id (module));
fprintf (fp, "\"%d\",", depth); /* depth */
/* output metrics */
print_csv_metric_block (fp, nmetrics);
/* recurse into nested sub-items */
if (iter->sub_list != NULL)
print_csv_sub_items (fp, iter->sub_list, module, i, totals, depth + 1);
free (nmetrics);
}
}
/* Output first-level items.
*
* On success, outputs item value. */
static void
print_csv_data (FILE *fp, GHolder *h, GPercTotals totals) {
GMetrics *nmetrics;
uint32_t i;
for (i = 0; i < h->idx; i++) {
set_data_metrics (h->items[i].metrics, &nmetrics, totals);
fprintf (fp, "\"%u\",", i); /* idx */
fprintf (fp, ","); /* no parent */
fprintf (fp, "\"%s\",", module_to_id (h->module));
/* output metrics */
print_csv_metric_block (fp, nmetrics);
if (h->sub_items_size)
print_csv_sub_items (fp, h->items[i].sub_list, h->module, i, totals, 1);
free (nmetrics);
}
}
#pragma GCC diagnostic ignored "-Wformat-nonliteral"
/* Output general statistics information. */
static void
print_csv_summary (FILE *fp) {
char now[DATE_TIME];
char *source = NULL;
const char *fmt;
int i = 0;
uint64_t total = 0;
uint32_t t = 0;
generate_time ();
strftime (now, DATE_TIME, "%Y-%m-%d %H:%M:%S %z", &now_tm);
/* generated date time */
fmt = "\"%d\",,\"%s\",,,,,,,,\"%s\",\"%s\"\r\n";
fprintf (fp, fmt, i++, GENER_ID, now, OVERALL_DATETIME);
/* total requests */
fmt = "\"%d\",,\"%s\",,,,,,,,\"%" PRIu64 "\",\"%s\"\r\n";
total = ht_get_processed ();
fprintf (fp, fmt, i++, GENER_ID, total, OVERALL_REQ);
/* valid requests */
fmt = "\"%d\",,\"%s\",,,,,,,,\"%" PRIu64 "\",\"%s\"\r\n";
total = ht_sum_valid ();
fprintf (fp, fmt, i++, GENER_ID, total, OVERALL_VALID);
/* invalid requests */
total = ht_get_invalid ();
fprintf (fp, fmt, i++, GENER_ID, total, OVERALL_FAILED);
/* generated time */
fmt = "\"%d\",,\"%s\",,,,,,,,\"%u\",\"%s\"\r\n";
t = ht_get_processing_time ();
fprintf (fp, fmt, i++, GENER_ID, t, OVERALL_GENTIME);
/* visitors */
fmt = "\"%d\",,\"%s\",,,,,,,,\"%" PRIu64 "\",\"%s\"\r\n";
total = ht_get_size_uniqmap (VISITORS);
fprintf (fp, fmt, i++, GENER_ID, total, OVERALL_VISITORS);
/* files */
total = ht_get_size_datamap (REQUESTS);
fprintf (fp, fmt, i++, GENER_ID, total, OVERALL_FILES);
/* excluded hits */
total = ht_get_excluded_ips ();
fprintf (fp, fmt, i++, GENER_ID, total, OVERALL_EXCL_HITS);
/* referrers */
total = ht_get_size_datamap (REFERRERS);
fprintf (fp, fmt, i++, GENER_ID, total, OVERALL_REF);
/* not found */
total = ht_get_size_datamap (NOT_FOUND);
fprintf (fp, fmt, i++, GENER_ID, total, OVERALL_NOTFOUND);
/* static files */
total = ht_get_size_datamap (REQUESTS_STATIC);
fprintf (fp, fmt, i++, GENER_ID, total, OVERALL_STATIC);
/* log size */
fmt = "\"%d\",,\"%s\",,,,,,,,\"%jd\",\"%s\"\r\n";
fprintf (fp, fmt, i++, GENER_ID, (intmax_t) get_log_sizes (), OVERALL_LOGSIZE);
/* bandwidth */
fmt = "\"%d\",,\"%s\",,,,,,,,\"%" PRIu64 "\",\"%s\"\r\n";
fprintf (fp, fmt, i++, GENER_ID, ht_sum_bw (), OVERALL_BANDWIDTH);
/* log path */
source = get_log_source_str (0);
fmt = "\"%d\",,\"%s\",,,,,,,,\"%s\",\"%s\"\r\n";
fprintf (fp, fmt, i++, GENER_ID, source, OVERALL_LOG);
free (source);
}
#pragma GCC diagnostic warning "-Wformat-nonliteral"
/* Entry point to generate a csv report writing it to the fp */
void
output_csv (GHolder *holder, const char *filename) {
GModule module;
GPercTotals totals;
const GPanel *panel = NULL;
size_t idx = 0;
FILE *fp;
fp = (filename != NULL) ? fopen (filename, "w") : stdout;
if (!fp)
FATAL ("Unable to open CSV file: %s.", strerror (errno));
if (!conf.no_csv_summary)
print_csv_summary (fp);
set_module_totals (&totals);
FOREACH_MODULE (idx, module_list) {
module = module_list[idx];
if (!(panel = panel_lookup (module))) {
continue;
}
panel->render (fp, holder + module, totals);
}
fclose (fp);
}
src/csv.h
+43
View File
@@ -0,0 +1,43 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#ifndef CSV_H_INCLUDED
#define CSV_H_INCLUDED
#include <stdio.h>
#include "parser.h"
#include "settings.h"
void output_csv (GHolder * holder, const char *filename);
#endif
src/dialogs.c
+1024
View File
File diff suppressed because it is too large Load Diff
src/dialogs.h
+45
View File
@@ -0,0 +1,45 @@
/**
* dialogs.h -- UI dialog windows
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef DIALOGS_H_INCLUDED
#define DIALOGS_H_INCLUDED
#include "ui.h"
#include "sort.h"
void load_help_popup (WINDOW * main_win);
void load_sort_win (WINDOW * main_win, GModule module, GSort * sort);
void load_schemes_win (WINDOW * main_win);
int render_confdlg (Logs * logs, GSpinner * spinner);
GAgents *load_host_agents (const char *addr);
void load_agent_list (WINDOW * main_win, char *addr);
void load_panels_win (WINDOW * main_win);
#endif
src/error.c
+278
View File
@@ -0,0 +1,278 @@
/**
* error.c -- error handling
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#if defined(__GLIBC__)
#include <execinfo.h>
#endif
#include <inttypes.h>
#include <sys/types.h>
#include <unistd.h>
#include "error.h"
#include "labels.h"
#include "parser.h"
static FILE *access_log;
static FILE *log_file;
static FILE *log_invalid;
static FILE *log_unknowns;
static Logs *log_data;
static struct sigaction old_sigsegv_handler;
/* Open a debug file whose name is specified in the given path. */
void
dbg_log_open (const char *path) {
if (path != NULL) {
log_file = fopen (path, "w");
if (log_file == NULL)
return;
}
}
/* Close the debug file. */
void
dbg_log_close (void) {
if (log_file != NULL)
fclose (log_file);
}
/* Open the invalid requests log file whose name is specified in the
* given path. */
void
invalid_log_open (const char *path) {
if (path != NULL) {
log_invalid = fopen (path, "w");
if (log_invalid == NULL)
return;
}
}
/* Close the invalid requests log file. */
void
invalid_log_close (void) {
if (log_invalid != NULL)
fclose (log_invalid);
}
/* Open the unknowns log file whose name is specified in the
* given path. */
void
unknowns_log_open (const char *path) {
if (path != NULL) {
log_unknowns = fopen (path, "w");
if (log_unknowns == NULL)
return;
}
}
/* Close the unknowns log file. */
void
unknowns_log_close (void) {
if (log_unknowns != NULL)
fclose (log_unknowns);
}
/* Set current overall parsed log data. */
void
set_signal_data (void *p) {
log_data = p;
}
/* Open the WebSocket access log file whose name is specified in the
* given path. */
int
access_log_open (const char *path) {
if (path == NULL)
return 0;
access_log = fopen (path, "a");
if (access_log == NULL)
return 1;
return 0;
}
/* Close the WebSocket access log file. */
void
access_log_close (void) {
if (access_log != NULL)
fclose (access_log);
}
/* Set up sigsegv handler. */
void
setup_sigsegv_handler (void) {
struct sigaction act;
sigemptyset (&act.sa_mask);
act.sa_flags = (int) SA_RESETHAND;
act.sa_handler = sigsegv_handler;
sigaction (SIGSEGV, &act, &old_sigsegv_handler);
}
static void
dump_struct_data (FILE *fp, GLog *glog, int pid) {
fprintf (fp, "==%d== FILE: %s\n", pid, glog->props.filename);
fprintf (fp, "==%d== Line number: %" PRIu64 "\n", pid, glog->processed);
fprintf (fp, "==%d== Invalid data: %" PRIu64 "\n", pid, glog->invalid);
fprintf (fp, "==%d== Piping: %d\n", pid, glog->piping);
fprintf (fp, "==%d==\n", pid);
}
/* Dump to the standard output the values of the overall parsed log
* data. */
static void
dump_struct (FILE *fp) {
int pid = getpid (), i;
if (!log_data)
return;
fprintf (fp, "==%d== VALUES AT CRASH POINT\n", pid);
fprintf (fp, "==%d==\n", pid);
for (i = 0; i < log_data->size; ++i)
dump_struct_data (fp, &log_data->glog[i], pid);
}
/* Custom SIGSEGV handler. */
void
sigsegv_handler (int sig) {
FILE *fp = stderr;
int pid = getpid ();
#if defined(__GLIBC__)
char **messages;
size_t size, i;
void *trace_stack[TRACE_SIZE];
#endif
(void) endwin ();
fprintf (fp, "\n");
fprintf (fp, "==%d== GoAccess %s crashed by Sig %d\n", pid, GO_VERSION, sig);
fprintf (fp, "==%d==\n", pid);
dump_struct (fp);
#if defined(__GLIBC__)
size = backtrace (trace_stack, TRACE_SIZE);
messages = backtrace_symbols (trace_stack, size);
fprintf (fp, "==%d== STACK TRACE:\n", pid);
fprintf (fp, "==%d==\n", pid);
for (i = 0; i < size; i++)
fprintf (fp, "==%d== %zu %s\n", pid, i, messages[i]);
#endif
fprintf (fp, "==%d==\n", pid);
fprintf (fp, "==%d== %s:\n", pid, ERR_PLEASE_REPORT);
fprintf (fp, "==%d== https://github.com/allinurl/goaccess/issues\n\n", pid);
fflush (fp);
/* Call old sigsegv handler; may be default exit or third party one (e.g. ASAN) */
sigaction (SIGSEGV, &old_sigsegv_handler, NULL);
}
/* Write formatted debug log data to the logfile. */
void
dbg_fprintf (const char *fmt, ...) {
va_list args;
if (!log_file)
return;
va_start (args, fmt);
vfprintf (log_file, fmt, args);
fflush (log_file);
va_end (args);
}
/* Write formatted invalid requests log data to the logfile. */
void
invalid_fprintf (const char *fmt, ...) {
va_list args;
if (!log_invalid)
return;
va_start (args, fmt);
vfprintf (log_invalid, fmt, args);
fflush (log_invalid);
va_end (args);
}
/* Write formatted unknown browsers/OSs log data to the logfile. */
void
unknowns_fprintf (const char *fmt, ...) {
va_list args;
if (!log_unknowns)
return;
va_start (args, fmt);
vfprintf (log_unknowns, fmt, args);
fflush (log_unknowns);
va_end (args);
}
/* Debug output */
void
dbg_printf (const char *fmt, ...) {
va_list args;
va_start (args, fmt);
vfprintf (stderr, fmt, args);
va_end (args);
}
/* Write formatted access log data to the logfile. */
void
access_fprintf (const char *fmt, ...) {
va_list args;
if (!access_log)
return;
va_start (args, fmt);
vfprintf (access_log, fmt, args);
fflush (access_log);
va_end (args);
}
src/error.h
+98
View File
@@ -0,0 +1,98 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef ERROR_H_INCLUDED
#define ERROR_H_INCLUDED
#ifndef COMMONS
#include "commons.h"
#endif
#ifdef HAVE_NCURSESW_NCURSES_H
#include <ncursesw/ncurses.h>
#elif HAVE_NCURSES_NCURSES_H
#include <ncurses/ncurses.h>
#elif HAVE_NCURSES_H
#include <ncurses.h>
#elif HAVE_CURSES_H
#include <curses.h>
#endif
#include <stdio.h>
#include "settings.h"
#define TRACE_SIZE 128
#define FATAL(fmt, ...) do { \
(void) endwin (); \
fprintf (stderr, "\nGoAccess - version %s - %s %s\n", GO_VERSION, __DATE__, __TIME__); \
fprintf (stderr, "Config file: %s\n", conf.iconfigfile ?: NO_CONFIG_FILE); \
fprintf (stderr, "\nFatal error has occurred"); \
fprintf (stderr, "\nError occurred at: %s - %s - %d\n", __FILE__, __FUNCTION__, __LINE__); \
fprintf (stderr, fmt, ##__VA_ARGS__); \
fprintf (stderr, "\n\n"); \
LOG_DEBUG ((fmt, ##__VA_ARGS__)); \
exit(EXIT_FAILURE); \
} while (0)
#ifdef DEBUG
#define DEBUG_TEST 1
#else
#define DEBUG_TEST 0
#endif
/* access requests log */
#define ACCESS_LOG(x, ...) do { access_fprintf x; } while (0)
/* debug log */
#define LOG_DEBUG(x, ...) do { dbg_fprintf x; } while (0)
/* invalid requests log */
#define LOG_INVALID(x, ...) do { invalid_fprintf x; } while (0)
/* unknown browser log */
#define LOG_UNKNOWNS(x, ...) do { unknowns_fprintf x; } while (0)
/* log debug wrapper */
#define LOG(x) do { if (DEBUG_TEST) dbg_printf x; } while (0)
int access_log_open (const char *path);
void access_fprintf (const char *fmt, ...) __attribute__((format (printf, 1, 2)));
void access_log_close (void);
void dbg_fprintf (const char *fmt, ...) __attribute__((format (printf, 1, 2)));
void dbg_log_close (void);
void dbg_log_open (const char *file);
void dbg_printf (const char *fmt, ...) __attribute__((format (printf, 1, 2)));
void invalid_fprintf (const char *fmt, ...) __attribute__((format (printf, 1, 2)));
void unknowns_fprintf (const char *fmt, ...) __attribute__((format (printf, 1, 2)));
void invalid_log_close (void);
void invalid_log_open (const char *path);
void set_signal_data (void *p);
void setup_sigsegv_handler (void);
void sigsegv_handler (int sig);
void unknowns_log_close (void);
void unknowns_log_open (const char *path);
#endif
src/fileio.c
+229
View File
@@ -0,0 +1,229 @@
/**
* fileio.c -- gFile I/O abstraction layer for goaccess
* This provides a unified interface for reading both regular and gzipped files
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include "fileio.h"
#ifdef HAVE_ZLIB
/* Check if a file is gzipped by examining its magic bytes */
static int
is_gzipped_file (const char *filename) {
FILE *fp;
unsigned char magic[2];
int result = 0;
if ((fp = fopen (filename, "rb")) == NULL)
return 0;
if (fread (magic, 1, 2, fp) == 2) {
/* gzip magic number is 0x1f 0x8b */
if (magic[0] == 0x1f && magic[1] == 0x8b)
result = 1;
}
fclose (fp);
return result;
}
#endif
/* Open a file for reading, automatically detecting gzip compression */
GFileHandle *
gfile_open (const char *filename, const char *mode) {
GFileHandle *fh;
if ((fh = calloc (1, sizeof (GFileHandle))) == NULL)
return NULL;
#ifdef HAVE_ZLIB
/* Check if file is gzipped */
if (is_gzipped_file (filename)) {
fh->is_gzipped = 1;
fh->gzfp = gzopen (filename, mode);
if (fh->gzfp == NULL) {
free (fh);
return NULL;
}
fh->fp = NULL;
return fh;
}
#endif
/* Open as regular file */
#ifdef HAVE_ZLIB
fh->is_gzipped = 0;
fh->gzfp = NULL;
#endif
fh->fp = fopen (filename, mode);
if (fh->fp == NULL) {
free (fh);
return NULL;
}
return fh;
}
/* Close a file handle */
void
gfile_close (GFileHandle *fh) {
if (!fh)
return;
#ifdef HAVE_ZLIB
if (fh->is_gzipped && fh->gzfp) {
gzclose (fh->gzfp);
} else
#endif
if (fh->fp) {
fclose (fh->fp);
}
free (fh);
}
/* Read a line from the file */
char *
gfile_gets (char *buf, int size, GFileHandle *fh) {
if (!fh || !buf || size <= 0)
return NULL;
#ifdef HAVE_ZLIB
if (fh->is_gzipped && fh->gzfp) {
return gzgets (fh->gzfp, buf, size);
}
#endif
if (fh->fp)
return fgets (buf, size, fh->fp);
return NULL;
}
/* Check if end of file is reached */
int
gfile_eof (GFileHandle *fh) {
if (!fh)
return 1;
#ifdef HAVE_ZLIB
if (fh->is_gzipped && fh->gzfp) {
return gzeof (fh->gzfp);
}
#endif
if (fh->fp)
return feof (fh->fp);
return 1;
}
/* Read data from file */
int
gfile_read (void *buf, size_t size, size_t count, GFileHandle *fh) {
if (!fh || !buf)
return 0;
#ifdef HAVE_ZLIB
if (fh->is_gzipped && fh->gzfp) {
int bytes_to_read = size * count;
int bytes_read = gzread (fh->gzfp, buf, bytes_to_read);
if (bytes_read < 0)
return 0;
return bytes_read / size;
}
#endif
if (fh->fp)
return fread (buf, size, count, fh->fp);
return 0;
}
/* Seek to a position in the file */
int
gfile_seek (GFileHandle *fh, long offset, int whence) {
if (!fh)
return -1;
#ifdef HAVE_ZLIB
if (fh->is_gzipped && fh->gzfp) {
/* gzseek returns the current offset, or -1 on error */
z_off_t result = gzseek (fh->gzfp, offset, whence);
return (result == -1) ? -1 : 0;
}
#endif
if (fh->fp)
return fseek (fh->fp, offset, whence);
return -1;
}
/* Get current position in file */
long
gfile_tell (GFileHandle *fh) {
if (!fh)
return -1;
#ifdef HAVE_ZLIB
if (fh->is_gzipped && fh->gzfp) {
return (long) gztell (fh->gzfp);
}
#endif
if (fh->fp)
return ftell (fh->fp);
return -1;
}
/* Check for file errors */
int
gfile_error (GFileHandle *fh) {
if (!fh)
return 1;
#ifdef HAVE_ZLIB
if (fh->is_gzipped && fh->gzfp) {
int errnum;
gzerror (fh->gzfp, &errnum);
return errnum != Z_OK && errnum != Z_STREAM_END;
}
#endif
if (fh->fp)
return ferror (fh->fp);
return 1;
}
src/fileio.h
+62
View File
@@ -0,0 +1,62 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef FILEIO_H_INCLUDED
#define FILEIO_H_INCLUDED
#include <stdio.h>
#ifdef HAVE_CONFIG_H
#include <config.h>
#endif
#ifdef HAVE_ZLIB
#include <zlib.h>
#endif
/* File handle abstraction */
typedef struct GFileHandle_ {
#ifdef HAVE_ZLIB
gzFile gzfp;
int is_gzipped;
#endif
FILE *fp;
} GFileHandle;
/* Function prototypes */
GFileHandle *gfile_open (const char *filename, const char *mode);
void gfile_close (GFileHandle * fh);
char *gfile_gets (char *buf, int size, GFileHandle * fh);
int gfile_eof (GFileHandle * fh);
int gfile_read (void *buf, size_t size, size_t count, GFileHandle * fh);
int gfile_seek (GFileHandle * fh, long offset, int whence);
long gfile_tell (GFileHandle * fh);
int gfile_error (GFileHandle * fh);
#endif /* FILEIO_H_INCLUDED */
src/gchart.c
+752
View File
@@ -0,0 +1,752 @@
/**
* gchart.c -- TUI chart implementation
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <math.h>
#include <inttypes.h>
#include <ctype.h>
#include "gchart.h"
#include "ui.h"
#include "color.h"
#include "commons.h"
#include "gholder.h"
#include "settings.h"
#include "xmalloc.h"
#define round(x) ((x) < 0 ? (int)((x) - 0.5) : (int)((x) + 0.5))
/* Formatting helpers */
char *
format_number_abbrev (uint64_t val, char *buf, size_t len) {
if (val >= 1000000000ULL) {
snprintf (buf, len, "%" PRIu64 "G", (uint64_t) ((val + 500000000ULL) / 1000000000ULL));
} else if (val >= 1000000ULL) {
snprintf (buf, len, "%" PRIu64 "M", (uint64_t) ((val + 500000ULL) / 1000000ULL));
} else if (val >= 1000ULL) {
snprintf (buf, len, "%" PRIu64 "K", (uint64_t) ((val + 500ULL) / 1000ULL));
} else {
snprintf (buf, len, "%" PRIu64, val);
}
buf[4] = '\0';
return buf;
}
static char *
format_bw_abbrev (uint64_t bytes, char *buf, size_t len) {
uint64_t v = bytes;
const char *s = "B";
if (bytes >= (1ULL << 40)) {
v = (bytes + (1ULL << 39)) >> 40;
s = "T";
} else if (bytes >= (1ULL << 30)) {
v = (bytes + (1ULL << 29)) >> 30;
s = "G";
} else if (bytes >= (1ULL << 20)) {
v = (bytes + (1ULL << 19)) >> 20;
s = "M";
} else if (bytes >= (1ULL << 10)) {
v = (bytes + (1ULL << 9)) >> 10;
s = "K";
} else {
v = bytes;
s = "B";
}
if (v >= 1000) {
snprintf (buf, len, "%" PRIu64, bytes);
} else {
snprintf (buf, len, "%" PRIu64 "%s", v, s);
}
buf[4] = '\0';
return buf;
}
static char *
format_time_abbrev (uint64_t us, char *buf, size_t len) {
uint64_t v = us;
const char *s = "\u00b5s";
if (us < 1000ULL) {
snprintf (buf, len, "%" PRIu64 "%s", us, s);
} else if (us < 1000000ULL) {
v = (us + 500ULL) / 1000ULL;
s = "ms";
snprintf (buf, len, "%" PRIu64 "%s", v, s);
} else if (us < 60000000ULL) {
v = (us + 500000ULL) / 1000000ULL;
s = "s";
snprintf (buf, len, "%" PRIu64 "%s", v, s);
} else if (us < 3600000000ULL) {
v = (us + 30000000ULL) / 60000000ULL;
s = "m";
snprintf (buf, len, "%" PRIu64 "%s", v, s);
} else if (us < 86400000000ULL) {
v = (us + 1800000000ULL) / 3600000000ULL;
s = "h";
snprintf (buf, len, "%" PRIu64 "%s", v, s);
} else {
v = (us + 43200000000ULL) / 86400000000ULL;
s = "d";
snprintf (buf, len, "%" PRIu64 "%s", v, s);
}
buf[5] = '\0';
return buf;
}
static char *
format_metric_value (uint64_t val, char *buf, size_t len, int metric_type) {
switch (metric_type) {
case CHART_METRIC_AVGTS:
case CHART_METRIC_CUMTS:
case CHART_METRIC_MAXTS:
return format_time_abbrev (val, buf, len);
case CHART_METRIC_BW:
return format_bw_abbrev (val, buf, len);
default:
return format_number_abbrev (val, buf, len);
}
}
static const char *
get_metric_name (int metric_type) {
switch (metric_type) {
case CHART_METRIC_HITS:
return MTRC_HITS_LBL;
case CHART_METRIC_VISITORS:
return MTRC_VISITORS_LBL;
case CHART_METRIC_BW:
return MTRC_BW_LBL;
case CHART_METRIC_AVGTS:
return MTRC_AVGTS_LBL;
case CHART_METRIC_CUMTS:
return MTRC_CUMTS_LBL;
case CHART_METRIC_MAXTS:
return MTRC_MAXTS_LBL;
default:
return MTRC_HITS_LBL;
}
}
/* Data helpers */
/* Check if a metric has data in a sub-list, recursing into nested sub-lists */
static int
sub_list_has_metric (GSubList *sl, int metric_type) {
GSubItem *sub;
GMetrics *m;
if (sl == NULL)
return 0;
for (sub = sl->head; sub; sub = sub->next) {
m = sub->metrics;
if (m) {
switch (metric_type) {
case CHART_METRIC_HITS:
if (m->hits > 0)
return 1;
break;
case CHART_METRIC_VISITORS:
if (m->visitors > 0)
return 1;
break;
case CHART_METRIC_BW:
if (m->nbw > 0)
return 1;
break;
case CHART_METRIC_AVGTS:
if (conf.serve_usecs && m->avgts.nts > 0)
return 1;
break;
case CHART_METRIC_CUMTS:
if (conf.serve_usecs && m->cumts.nts > 0)
return 1;
break;
case CHART_METRIC_MAXTS:
if (conf.serve_usecs && m->maxts.nts > 0)
return 1;
break;
default:
if (m->hits > 0)
return 1;
break;
}
}
if (sub->sub_list != NULL && sub_list_has_metric (sub->sub_list, metric_type))
return 1;
}
return 0;
}
int
metric_has_data (GHolder *h, int metric_type) {
uint32_t i;
int has = 0;
GMetrics *m;
if (!h || h->idx == 0)
return 0;
for (i = 0; i < h->idx; i++) {
m = h->items[i].metrics;
if (m) {
switch (metric_type) {
case CHART_METRIC_HITS:
if (m->hits > 0)
has = 1;
break;
case CHART_METRIC_VISITORS:
if (m->visitors > 0)
has = 1;
break;
case CHART_METRIC_BW:
if (m->nbw > 0)
has = 1;
break;
case CHART_METRIC_AVGTS:
if (conf.serve_usecs && m->avgts.nts > 0)
has = 1;
break;
case CHART_METRIC_CUMTS:
if (conf.serve_usecs && m->cumts.nts > 0)
has = 1;
break;
case CHART_METRIC_MAXTS:
if (conf.serve_usecs && m->maxts.nts > 0)
has = 1;
break;
default:
if (m->hits > 0)
has = 1;
break;
}
if (has)
return 1;
}
if (h->module != HOSTS && h->items[i].sub_list) {
if (sub_list_has_metric (h->items[i].sub_list, metric_type))
return 1;
}
}
return 0;
}
int
get_available_metrics (GModule module, int metrics[CHART_METRIC_COUNT]) {
int count = 0;
const GOutput *out = output_lookup (module);
if (!out)
return 0;
if (out->hits)
metrics[count++] = CHART_METRIC_HITS;
if (out->visitors)
metrics[count++] = CHART_METRIC_VISITORS;
if (out->bw && conf.bandwidth)
metrics[count++] = CHART_METRIC_BW;
if (out->avgts && conf.serve_usecs)
metrics[count++] = CHART_METRIC_AVGTS;
if (out->cumts && conf.serve_usecs)
metrics[count++] = CHART_METRIC_CUMTS;
if (out->maxts && conf.serve_usecs)
metrics[count++] = CHART_METRIC_MAXTS;
return count;
}
static uint64_t
get_metric_value (GMetrics *metrics, int metric_type) {
if (!metrics)
return 0;
switch (metric_type) {
case CHART_METRIC_HITS:
return metrics->hits;
case CHART_METRIC_VISITORS:
return metrics->visitors;
case CHART_METRIC_BW:
return metrics->nbw;
case CHART_METRIC_AVGTS:
return conf.serve_usecs ? metrics->avgts.nts : 0;
case CHART_METRIC_CUMTS:
return conf.serve_usecs ? metrics->cumts.nts : 0;
case CHART_METRIC_MAXTS:
return conf.serve_usecs ? metrics->maxts.nts : 0;
default:
return metrics->hits;
}
}
/* Chart item flattening */
typedef struct {
GMetrics *metrics;
int is_subitem;
int flat_idx;
} ChartItem;
/* Recursively count all sub-items in a sub-list tree (chart-local). */
static int
count_sub_items_recursive_chart (GSubList *sl) {
GSubItem *iter;
int count = 0;
if (sl == NULL)
return 0;
for (iter = sl->head; iter; iter = iter->next) {
count++;
if (iter->sub_list)
count += count_sub_items_recursive_chart (iter->sub_list);
}
return count;
}
/* Count visible sub-items recursively given per-node expand state (chart-local). */
static int
count_visible_sub_chart (GSubList *sl, const uint8_t *node_exp, int node_exp_size, int *full_idx) {
GSubItem *iter;
int count = 0;
if (sl == NULL)
return 0;
for (iter = sl->head; iter; iter = iter->next) {
int my_full = *full_idx;
count++;
(*full_idx)++;
if (iter->sub_list && iter->sub_list->size > 0) {
int expanded = 1;
if (node_exp != NULL && my_full < node_exp_size)
expanded = node_exp[my_full];
if (expanded)
count += count_visible_sub_chart (iter->sub_list, node_exp, node_exp_size, full_idx);
else
*full_idx += count_sub_items_recursive_chart (iter->sub_list);
}
}
return count;
}
/* Recursively flatten visible sub-items into the chart items array */
static void
flatten_sub_items (GSubList *sl, ChartItem *items, int *idx, int depth, int *full_idx,
const uint8_t *node_exp, int node_exp_size) {
GSubItem *sub;
if (sl == NULL)
return;
for (sub = sl->head; sub; sub = sub->next) {
int my_full = *full_idx;
items[*idx] = (ChartItem) {
.metrics = sub->metrics,.is_subitem = depth,.flat_idx = *idx};
(*idx)++;
(*full_idx)++;
if (sub->sub_list != NULL && sub->sub_list->size > 0) {
int expanded = 1;
if (node_exp != NULL && my_full < node_exp_size)
expanded = node_exp[my_full];
if (expanded)
flatten_sub_items (sub->sub_list, items, idx, depth + 1, full_idx, node_exp, node_exp_size);
else
*full_idx += count_sub_items_recursive_chart (sub->sub_list);
}
}
}
static int
build_chart_items (GHolder *h, ChartItem **out, const uint8_t *item_expanded,
int item_expanded_size) {
uint32_t i;
int count = h->idx, idx = 0, full_idx = 0;
ChartItem *items;
/* Only include sub-items if we have expand state (i.e., panel is expanded) */
if (h->module != HOSTS && item_expanded != NULL && item_expanded_size > 0) {
/* Count visible items using per-node expand state */
int fi = 0;
for (i = 0; i < h->idx; i++) {
int my_full = fi;
fi++;
if (h->items[i].sub_list && h->items[i].sub_list->size > 0) {
int expanded = 1;
if (my_full < item_expanded_size)
expanded = item_expanded[my_full];
if (expanded)
count +=
count_visible_sub_chart (h->items[i].sub_list, item_expanded, item_expanded_size, &fi);
else
fi += count_sub_items_recursive_chart (h->items[i].sub_list);
}
}
}
/* If item_expanded is NULL, we only render root items (count = h->idx already) */
items = xcalloc (count, sizeof (ChartItem));
for (i = 0; i < h->idx; i++) {
int my_full = full_idx;
items[idx] = (ChartItem) {
.metrics = h->items[i].metrics,.is_subitem = 0,.flat_idx = idx};
idx++;
full_idx++;
/* Only flatten sub-items if panel is expanded (item_expanded != NULL) */
if (h->module != HOSTS && item_expanded != NULL && h->items[i].sub_list &&
h->items[i].sub_list->size > 0) {
int should_expand = 1;
if (my_full < item_expanded_size)
should_expand = item_expanded[my_full];
if (should_expand)
flatten_sub_items (h->items[i].sub_list, items, &idx, 1, &full_idx, item_expanded,
item_expanded_size);
else
full_idx += count_sub_items_recursive_chart (h->items[i].sub_list);
}
}
*out = items;
return idx;
}
static uint64_t
apply_log_scale (uint64_t value, uint64_t max_value) {
double log_val, log_max;
if (value == 0 || max_value == 0)
return 0;
log_val = log10 ((double) value + 1.0);
log_max = log10 ((double) max_value + 1.0);
if (log_max == 0)
return value;
return (uint64_t) ((log_val / log_max) * max_value);
}
/* Drawing primitives */
static void
draw_vbar (ChartDrawCtx *ctx, int x, int filled, GColors *color, int is_selected,
uint64_t actual_val) {
WINDOW *w = ctx->win;
int y_top = ctx->y_chart_start;
int width = ctx->bar_width;
int i, y, ly, lx;
char buf[16];
GColors *val_col;
for (i = 0; i < ctx->bar_height; i++)
mvwhline (w, y_top + i, x, ' ', width);
if (is_selected) {
/* Use COLOR_SELECTED colors + underline + bold for selected bar */
GColors *sel = get_color (COLOR_SELECTED);
wattron (w, COLOR_PAIR (sel->pair->idx) | A_BOLD | A_UNDERLINE);
} else {
/* Normal bar: original color + reverse */
wattron (w, COLOR_PAIR (color->pair->idx) | color->attr | A_REVERSE);
}
/* Draw filled part */
for (i = 0; i < filled; i++) {
y = y_top + ctx->bar_height - 1 - i;
mvwhline (w, y, x, ' ', width);
}
/* Turn off what we turned on */
wattroff (w, COLOR_PAIR (color->pair->idx) | color->attr | A_REVERSE | A_BOLD | A_UNDERLINE);
/* Value label (unchanged) */
if (actual_val > 0 && ctx->show_bar_values) {
format_metric_value (actual_val, buf, sizeof (buf), ctx->metric_type);
y = y_top + ctx->bar_height - filled;
ly = y - 1;
if (ly < y_top - 1)
ly = y_top - 1;
val_col = get_color (COLOR_CHART_VALUES);
wattron (w, COLOR_PAIR (val_col->pair->idx) | val_col->attr);
if (is_selected)
wattron (w, A_BOLD);
lx = x + (width - (int) strlen (buf)) / 2;
if (lx < x)
lx = x;
mvwprintw (w, ly, lx, "%s", buf);
wattroff (w, COLOR_PAIR (val_col->pair->idx) | val_col->attr);
if (is_selected)
wattroff (w, A_BOLD);
}
}
static void
draw_axes (ChartDrawCtx *ctx) {
WINDOW *w;
int ys, xs, h, i, yt, lx;
GColors *axis;
uint64_t maxv;
char buf[16];
const int ticks = 4;
if (!ctx->show_axes)
return;
w = ctx->win;
ys = ctx->y_chart_start;
xs = ctx->x_start;
h = ctx->bar_height;
axis = get_color (COLOR_CHART_AXIS);
maxv = ctx->display_max;
wattron (w, axis->attr | COLOR_PAIR (axis->pair->idx));
for (i = 0; i <= h; i++)
mvwaddch (w, ys + i, xs - 2, '|');
for (i = 0; i < getmaxx (w) - xs - 2; i++)
mvwaddch (w, ys + h, xs + i, '-');
for (i = 0; i <= ticks; i++) {
uint64_t v = (maxv * (uint64_t) i + ticks / 2) / ticks;
yt = ys + h - ((v * (int64_t) h + maxv / 2) / maxv);
format_metric_value (v, buf, sizeof (buf), ctx->metric_type);
lx = xs - 3 - (int) strlen (buf);
if (lx < 0)
lx = 0;
mvwprintw (w, yt, lx, "%s", buf);
mvwaddch (w, yt, xs - 2, '+');
}
wattroff (w, axis->attr | COLOR_PAIR (axis->pair->idx));
}
static void
draw_chart_indicator (ChartDrawCtx *ctx) {
char ind[96], mname[64];
GColors *col = get_color (COLOR_CHART_AXIS);
strncpy (mname, get_metric_name (ctx->metric_type), sizeof (mname) - 1);
mname[sizeof (mname) - 1] = '\0';
for (char *p = mname; *p; p++)
*p = toupper ((unsigned char) *p);
snprintf (ind, sizeof (ind), "[%s:%s%s]",
mname, ctx->use_log_scale ? "LOG" : "LINEAR", ctx->reverse_bars ? ":REV" : "");
wattron (ctx->win, col->attr | COLOR_PAIR (col->pair->idx));
mvwprintw (ctx->win, ctx->y_start, 1, "%s", ind);
wattroff (ctx->win, col->attr | COLOR_PAIR (col->pair->idx));
}
/* Window & selection helpers */
static int
get_chart_selected_root (const ChartDrawCtx *ctx) {
int cum = 0, subsz = 0;
uint32_t j;
if (ctx->selected_idx < 0)
return -1;
if (ctx->holder->module != HOSTS)
return ctx->selected_idx;
for (j = 0; j < ctx->holder->idx; j++) {
subsz = ctx->holder->items[j].sub_list ? ctx->holder->items[j].sub_list->size : 0;
if (ctx->selected_idx == cum)
return j;
cum += 1 + subsz;
}
return -1;
}
static void
compute_bar_window (ChartDrawCtx *ctx, int num_items) {
int start = ctx->scroll_offset;
int vis = ctx->visible_bars;
int sel = get_chart_selected_root (ctx);
if (sel >= 0 && num_items > vis) {
if (sel < start)
start = sel;
else if (sel >= start + vis)
start = sel - vis + 1;
}
if (start < 0)
start = 0;
if (num_items > vis && start > num_items - vis)
start = num_items - vis;
if (num_items <= vis)
start = 0;
ctx->start_bar = start;
ctx->end_bar = start + vis;
if (ctx->end_bar > num_items)
ctx->end_bar = num_items;
}
static uint64_t
compute_local_max (ChartItem *items, const ChartDrawCtx *ctx) {
uint64_t mx = 0;
for (int i = ctx->start_bar; i < ctx->end_bar; i++) {
uint64_t v = get_metric_value (items[i].metrics, ctx->metric_type);
if (v > mx)
mx = v;
}
return mx;
}
static void
compute_chart_window (ChartDrawCtx *ctx, ChartItem *items) {
compute_bar_window (ctx, ctx->num_items);
ctx->display_max = compute_local_max (items, ctx);
}
static void
draw_chart_bars (ChartDrawCtx *ctx, ChartItem *items) {
GColors *bar_c = get_color (COLOR_BARS), *c = NULL;
GColors *sub_c = get_color (COLOR_SUBBARS);
int sel_root = get_chart_selected_root (ctx);
int i, x, data_idx, vis_pos, is_sel;
uint64_t val, dval;
int filled;
for (i = ctx->start_bar; i < ctx->end_bar; i++) {
data_idx = ctx->reverse_bars ? (ctx->end_bar - 1 - (i - ctx->start_bar)) : i;
if (data_idx >= ctx->num_items)
continue;
val = get_metric_value (items[data_idx].metrics, ctx->metric_type);
dval = ctx->use_log_scale ? apply_log_scale (val, ctx->display_max) : val;
filled = (int) ((dval * (int64_t) ctx->bar_height + ctx->display_max / 2) / ctx->display_max);
if (filled > ctx->bar_height)
filled = ctx->bar_height;
vis_pos = i - ctx->start_bar;
x = ctx->x_start + vis_pos * (ctx->bar_width + ctx->bar_gap);
is_sel = (items[data_idx].flat_idx == sel_root);
c = items[data_idx].is_subitem ? sub_c : bar_c;
draw_vbar (ctx, x, filled, c, is_sel, ctx->show_bar_values ? val : 0);
}
}
static void
draw_chart_caret (ChartDrawCtx *ctx, ChartItem *items) {
int sel_root = get_chart_selected_root (ctx);
if (sel_root < 0)
return;
for (int i = ctx->start_bar; i < ctx->end_bar; i++) {
int didx = ctx->reverse_bars ? (ctx->end_bar - 1 - (i - ctx->start_bar)) : i;
if (didx >= ctx->num_items)
continue;
if (items[didx].flat_idx == sel_root) {
int pos = i - ctx->start_bar;
int x = ctx->x_start + pos * (ctx->bar_width + ctx->bar_gap) + ctx->bar_width / 2;
mvwaddch (ctx->win, ctx->y_start + ctx->chart_height, x, '^');
break;
}
}
}
static void
check_top_padding (ChartDrawCtx *ctx, ChartItem *items) {
ctx->needs_top_padding = 0;
for (int i = ctx->start_bar; i < ctx->end_bar; i++) {
uint64_t v = get_metric_value (items[i].metrics, ctx->metric_type);
uint64_t dv = ctx->use_log_scale ? apply_log_scale (v, ctx->display_max) : v;
int f = (int) ((dv * (int64_t) ctx->bar_height + ctx->display_max / 2) / ctx->display_max);
if (f >= ctx->bar_height) {
ctx->needs_top_padding = 1;
break;
}
}
}
/* Main function */
void
draw_panel_chart (WINDOW *win, GHolder *h, const ChartDrawCtx *user_opts) {
uint64_t global_max = 0, v = 0;
int i = 0, y = 0, height = 0, screen_w = 0;
ChartDrawCtx ctx = *user_opts;
ChartItem *items = NULL;
ctx.win = win;
ctx.holder = h;
ctx.x_start = ctx.show_axes ? 8 : 2;
ctx.num_items = build_chart_items (h, &items, ctx.item_expanded, ctx.item_expanded_size);
if (ctx.num_items == 0) {
free (items);
return;
}
/* Early exit if no data for this metric */
for (i = 0; i < ctx.num_items; i++) {
v = get_metric_value (items[i].metrics, ctx.metric_type);
if (v > global_max)
global_max = v;
}
if (global_max == 0) {
free (items);
return;
}
draw_chart_indicator (&ctx);
y = ctx.y_start + 1;
height = ctx.chart_height - 1;
ctx.bar_height = height - 1;
screen_w = getmaxx (win) - ctx.x_start - 2;
if (ctx.auto_width) {
ctx.bar_gap = 1;
ctx.bar_width = (screen_w - (ctx.num_items - 1) * ctx.bar_gap) / ctx.num_items;
if (ctx.bar_width < 1)
ctx.bar_width = 1;
} else {
ctx.bar_width = BAR_W;
ctx.bar_gap = BAR_GAP;
}
ctx.visible_bars = screen_w / (ctx.bar_width + ctx.bar_gap);
if (ctx.visible_bars < 1)
ctx.visible_bars = 1;
compute_chart_window (&ctx, items);
check_top_padding (&ctx, items);
ctx.y_chart_start = y + (ctx.needs_top_padding ? 1 : 0);
draw_chart_bars (&ctx, items);
draw_axes (&ctx);
draw_chart_caret (&ctx, items);
free (items);
}
src/gchart.h
+93
View File
@@ -0,0 +1,93 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef GCHART_H_INCLUDED
#define GCHART_H_INCLUDED
#include <ncurses.h>
#include "commons.h"
/* Forward declarations */
typedef struct GHolder_ GHolder;
#define DASH_CHART_HEIGHT 7
#define BAR_W 5
#define BAR_GAP 1
typedef enum {
CHART_METRIC_HITS = 0,
CHART_METRIC_VISITORS,
CHART_METRIC_BW,
CHART_METRIC_AVGTS,
CHART_METRIC_CUMTS,
CHART_METRIC_MAXTS,
CHART_METRIC_COUNT
} GChartMetric;
/* All drawing options + computed state in one place */
typedef struct {
WINDOW *win;
GHolder *holder;
int y_start;
int chart_height;
int auto_width;
int selected_idx;
int scroll_offset;
int show_axes;
int show_bar_values;
int metric_type;
int use_log_scale;
int reverse_bars;
/* Computed values */
int x_start;
int bar_width;
int bar_gap;
int visible_bars;
int start_bar;
int end_bar;
uint64_t display_max;
int bar_height;
int y_chart_start;
int needs_top_padding;
int num_items; /* total number of chart items */
/* Per-item expand state for filtering visible chart bars */
const uint8_t *item_expanded; /* NULL = all expanded, else per-root-item state */
int item_expanded_size; /* number of entries in item_expanded */
} ChartDrawCtx;
/* Main drawing function */
void draw_panel_chart (WINDOW * win, GHolder * h, const ChartDrawCtx * opts);
int metric_has_data (GHolder * h, int metric_type);
int get_available_metrics (GModule module, int metrics[CHART_METRIC_COUNT]);
char *format_number_abbrev (uint64_t val, char *buf, size_t len);
#endif
src/gdashboard.c
+1781
View File
File diff suppressed because it is too large Load Diff
src/gdashboard.h
+168
View File
@@ -0,0 +1,168 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#ifndef GDASHBOARD_H_INCLUDED
#define GDASHBOARD_H_INCLUDED
#include <stdint.h>
#include "ui.h"
/* IMPORTANT: These defines are BASE VALUES used for calculation.
* Actual positions are calculated dynamically in render_content() */
#define DASH_HEAD_POS 0 /* position of header line */
#define DASH_EMPTY_POS 1 /* empty line position */
/* Chart positioning - these are fixed */
#define DASH_CHART_START 2 /* first chart row */
#define DASH_CHART_END (DASH_CHART_START + DASH_CHART_HEIGHT - 1) /* last chart row */
/* These are reference values - actual positions calculated dynamically */
#define DASH_BLANK1_POS (DASH_CHART_END + 1) /* blank after chart (for caret) */
#define DASH_BLANK2_POS (DASH_CHART_END + 2) /* blank before cols (if cols enabled) */
#define DASH_COLS_POS (DASH_CHART_END + 3) /* position of column names (if enabled) */
#define DASH_DASHES_POS (DASH_CHART_END + 4) /* position of dashes (if enabled) */
#define DASH_DATA_POS (DASH_CHART_END + 5) /* data line position (with cols) */
/* Helper macros for calculations */
#define DASH_NON_DATA (DASH_DATA_POS) /* number of rows without data stats */
#define DASH_COL_ROWS 2 /* number of rows for column values + dashed lines */
/* Panel sizes - these determine how many rows each panel occupies
* Structure:
* With columns: header + blank + chart(7) + blank + blank + cols + dashes + data(N) + blank = 13 + N
* Without columns: header + blank + chart(7) + blank + data(N) + blank = 11 + N
*/
#define DASH_COLLAPSED (DASH_DATA_POS + 8 + 1) /* +1 for trailing blank */
#define DASH_EXPANDED (DASH_DATA_POS + 28 + 1) /* +1 for trailing blank */
/* Other constants */
#define DASH_INIT_X 1 /* start position (x-axis) */
#define DASH_BW_LEN 11 /* max bandwidth string length */
#define DASH_SRV_TM_LEN 9 /* max time served length */
#define DASH_SPACE 1 /* space between columns (metrics) */
typedef struct {
int header_pos;
int blank_after_header;
int chart_start;
int chart_end;
int blank_for_caret;
int blank_before_cols;
int cols_header_pos;
int cols_dashes_pos;
int data_start;
int data_end;
} PanelLayout;
/* Common render data line fields */
typedef struct GDashRender_ {
WINDOW *win;
int y;
int w;
int idx;
int sel;
} GDashRender;
/* Dashboard panel item */
typedef struct GDashData_ {
GMetrics *metrics;
short is_subitem;
short has_children; /* 1 if this node has sub-items */
int node_full_idx; /* index into node_expanded[] (full DFS position) */
} GDashData;
/* Dashboard panel meta data */
typedef struct GDashMeta_ {
uint64_t max_hits; /* maximum value on the hits column */
uint64_t max_visitors; /* maximum value on the visitors column */
uint64_t max_bw;
/* determine the maximum metric's length of these metrics */
/* for instance, 1022 is the max value for the hits column and its length = 4 */
int hits_len;
int hits_perc_len;
int visitors_len;
int visitors_perc_len;
int bw_len;
int bw_perc_len;
int avgts_len;
int cumts_len;
int maxts_len;
int method_len;
int protocol_len;
int data_len;
} GDashMeta;
/* Dashboard panel */
typedef struct GDashModule_ {
GDashData *data; /* data metrics */
GModule module; /* module */
GDashMeta meta; /* meta data */
const char *head; /* panel header */
const char *desc; /* panel description */
uint32_t alloc_data; /* number of data items allocated. */
/* e.g., MAX_CHOICES or holder size */
int dash_size; /* dashboard size */
uint32_t holder_size; /* hash table size */
uint32_t ht_size; /* hash table size */
int idx_data; /* idx data */
unsigned short pos_y; /* dashboard current Y position */
} GDashModule;
/* Dashboard */
typedef struct GDash_ {
int total_alloc; /* number of allocated dashboard lines */
GDashModule module[TOTAL_MODULES];
} GDash;
/* Function Prototypes */
GDashData *new_gdata (uint32_t size);
GDash *new_gdash (void);
uint32_t get_num_collapsed_data_rows (void);
uint32_t get_num_expanded_data_rows (void);
int perform_next_find (GHolder * h, GScroll * scroll);
int render_find_dialog (WINDOW * main_win, GScroll * scroll);
int set_module_from_mouse_event (GScroll * scroll, GDash * dash, int y);
uint32_t get_ht_size_by_module (GModule module);
void display_content (WINDOW * win, GDash * dash, GScroll * gscroll, GHolder * holder);
void free_dashboard (GDash * dash);
void load_data_to_dash (GHolder * h, GDash * dash, GModule module, GScroll * scroll);
void reset_find (void);
void reset_scroll_offsets (GScroll * scroll);
#endif
src/gdns.c
+276
View File
@@ -0,0 +1,276 @@
/**
* gdns.c -- hosts resolver
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#define _MULTI_THREADED
#ifdef __FreeBSD__
#include <sys/socket.h>
#endif
#if HAVE_CONFIG_H
#include <config.h>
#endif
#include <pthread.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <ctype.h>
#include <netdb.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <time.h>
#include "gdns.h"
#include "error.h"
#include "gkhash.h"
#include "goaccess.h"
#include "util.h"
#include "xmalloc.h"
GDnsThread gdns_thread;
static GDnsQueue *gdns_queue;
/* Initialize the queue. */
void
gqueue_init (GDnsQueue *q, int capacity) {
q->head = 0;
q->tail = -1;
q->size = 0;
q->capacity = capacity;
}
/* Get the current size of queue.
*
* Returns the size of the queue. */
int
gqueue_size (GDnsQueue *q) {
return q->size;
}
/* Determine if the queue is empty.
*
* Returns true if empty, otherwise false. */
int
gqueue_empty (GDnsQueue *q) {
return q->size == 0;
}
/* Determine if the queue is full.
*
* Returns true if full, otherwise false. */
int
gqueue_full (GDnsQueue *q) {
return q->size == q->capacity;
}
/* Free the queue. */
void
gqueue_destroy (GDnsQueue *q) {
free (q);
}
/* Add at the end of the queue a string item.
*
* If the queue is full, -1 is returned.
* If added to the queue, 0 is returned. */
int
gqueue_enqueue (GDnsQueue *q, const char *item) {
if (gqueue_full (q))
return -1;
q->tail = (q->tail + 1) % q->capacity;
strncpy (q->buffer[q->tail], item, sizeof (q->buffer[q->tail]));
q->buffer[q->tail][sizeof (q->buffer[q->tail]) - 1] = '\0';
q->size++;
return 0;
}
/* Find a string item in the queue.
*
* If the queue is empty, or the item is not in the queue, 0 is returned.
* If found, 1 is returned. */
int
gqueue_find (GDnsQueue *q, const char *item) {
int i;
if (gqueue_empty (q))
return 0;
for (i = 0; i < q->size; i++) {
if (strcmp (item, q->buffer[i]) == 0)
return 1;
}
return 0;
}
/* Remove a string item from the head of the queue.
*
* If the queue is empty, NULL is returned.
* If removed, the string item is returned. */
char *
gqueue_dequeue (GDnsQueue *q) {
char *item;
if (gqueue_empty (q))
return NULL;
item = q->buffer[q->head];
q->head = (q->head + 1) % q->capacity;
q->size--;
return item;
}
/* Get the corresponding hostname given an IP address.
*
* On error, a string error message is returned.
* On success, a malloc'd hostname is returned. */
static char *
reverse_host (const struct sockaddr *a, socklen_t length) {
char h[H_SIZE] = { 0 };
int flags, st;
flags = NI_NAMEREQD;
st = getnameinfo (a, length, h, H_SIZE, NULL, 0, flags);
if (!st) {
/* BSD returns \0 while Linux . on solve lookups */
if (*h == '\0')
return alloc_string (".");
return alloc_string (h);
}
return alloc_string (gai_strerror (st));
}
/* Determine if IPv4 or IPv6 and resolve.
*
* On error, NULL is returned.
* On success, a malloc'd hostname is returned. */
char *
reverse_ip (char *str) {
union {
struct sockaddr addr;
struct sockaddr_in6 addr6;
struct sockaddr_in addr4;
} a;
if (str == NULL || *str == '\0')
return NULL;
memset (&a, 0, sizeof (a));
if (1 == inet_pton (AF_INET, str, &a.addr4.sin_addr)) {
a.addr4.sin_family = AF_INET;
return reverse_host (&a.addr, sizeof (a.addr4));
} else if (1 == inet_pton (AF_INET6, str, &a.addr6.sin6_addr)) {
a.addr6.sin6_family = AF_INET6;
return reverse_host (&a.addr, sizeof (a.addr6));
}
return NULL;
}
/* Producer - Resolve an IP address and add it to the queue. */
void
dns_resolver (char *addr) {
pthread_mutex_lock (&gdns_thread.mutex);
/* queue is not full and the IP address is not in the queue */
if (!gqueue_full (gdns_queue) && !gqueue_find (gdns_queue, addr)) {
/* add the IP to the queue */
gqueue_enqueue (gdns_queue, addr);
pthread_cond_broadcast (&gdns_thread.not_empty);
}
pthread_mutex_unlock (&gdns_thread.mutex);
}
/* Consumer - Once an IP has been resolved, add it to dwithe hostnames
* hash structure. */
static void
dns_worker (void GO_UNUSED (*ptr_data)) {
char *ip = NULL, *host = NULL;
while (1) {
pthread_mutex_lock (&gdns_thread.mutex);
/* wait until an item has been added to the queue */
while (gqueue_empty (gdns_queue))
pthread_cond_wait (&gdns_thread.not_empty, &gdns_thread.mutex);
ip = gqueue_dequeue (gdns_queue);
pthread_mutex_unlock (&gdns_thread.mutex);
host = reverse_ip (ip);
pthread_mutex_lock (&gdns_thread.mutex);
if (!active_gdns) {
pthread_mutex_unlock (&gdns_thread.mutex);
free (host);
return;
}
/* insert the corresponding IP -> hostname map */
if (host != NULL && active_gdns) {
ht_insert_hostname (ip, host);
free (host);
}
pthread_cond_signal (&gdns_thread.not_full);
pthread_mutex_unlock (&gdns_thread.mutex);
}
}
/* Initialize queue and dns thread */
void
gdns_init (void) {
gdns_queue = xmalloc (sizeof (GDnsQueue));
gqueue_init (gdns_queue, QUEUE_SIZE);
if (pthread_cond_init (&(gdns_thread.not_empty), NULL))
FATAL ("Failed init thread condition");
if (pthread_cond_init (&(gdns_thread.not_full), NULL))
FATAL ("Failed init thread condition");
if (pthread_mutex_init (&(gdns_thread.mutex), NULL))
FATAL ("Failed init thread mutex");
}
/* Destroy (free) queue */
void
gdns_free_queue (void) {
gqueue_destroy (gdns_queue);
}
/* Create a DNS thread and make it active */
void
gdns_thread_create (void) {
int th;
active_gdns = 1;
th = pthread_create (&(gdns_thread.thread), NULL, (void *) &dns_worker, NULL);
if (th)
FATAL ("Return code from pthread_create(): %d", th);
pthread_detach (gdns_thread.thread);
}
src/gdns.h
+68
View File
@@ -0,0 +1,68 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef GDNS_H_INCLUDED
#define GDNS_H_INCLUDED
#define H_SIZE 1025
#define QUEUE_SIZE 400
typedef struct GDnsThread_ {
pthread_cond_t not_empty; /* not empty queue condition */
pthread_cond_t not_full; /* not full queue condition */
pthread_mutex_t mutex;
pthread_t thread;
} GDnsThread;
typedef struct GDnsQueue_ {
int head; /* index to head of queue */
int tail; /* index to tail of queue */
int size; /* queue size */
int capacity; /* length at most */
char buffer[QUEUE_SIZE][H_SIZE]; /* data item */
} GDnsQueue;
extern GDnsThread gdns_thread;
char *gqueue_dequeue (GDnsQueue * q);
char *reverse_ip (char *str);
int gqueue_empty (GDnsQueue * q);
int gqueue_enqueue (GDnsQueue * q, const char *item);
int gqueue_find (GDnsQueue * q, const char *item);
int gqueue_full (GDnsQueue * q);
int gqueue_size (GDnsQueue * q);
void dns_resolver (char *addr);
void gdns_free_queue (void);
void gdns_init (void);
void gdns_queue_free (void);
void gdns_thread_create (void);
void gqueue_destroy (GDnsQueue * q);
void gqueue_init (GDnsQueue * q, int capacity);
#endif
src/geoip1.c
+610
View File
@@ -0,0 +1,610 @@
/**
* geoip.c -- implementation of GeoIP (legacy)
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#ifdef HAVE_LIBGEOIP
#include <GeoIP.h>
#include <GeoIPCity.h>
#endif
#include "geoip1.h"
#include "error.h"
#include "util.h"
static GeoIP **geoips = NULL;
static GeoIP *geo_location_data = NULL;
static int db_cnt = 0;
static int legacy_db = 0;
/* Determine if we have a valid geoip resource.
*
* If the geoip resource is NULL, 0 is returned.
* If the geoip resource is valid and malloc'd, 1 is returned. */
int
is_geoip_resource (void) {
return ((geoips && db_cnt) || (legacy_db && geo_location_data));
}
/* Free up GeoIP resources */
void
geoip_free (void) {
int idx = 0;
if (!is_geoip_resource ())
return;
for (idx = 0; idx < db_cnt; idx++)
GeoIP_delete (geoips[idx]);
if (legacy_db)
GeoIP_delete (geo_location_data);
GeoIP_cleanup ();
free (geoips);
geoips = NULL;
}
/* Open the given GeoLocation database and set its charset.
*
* On error, it aborts.
* On success, a new geolocation structure is returned. */
static GeoIP *
geoip_open_db (const char *db) {
GeoIP *geoip = GeoIP_open (db, GEOIP_MEMORY_CACHE);
if (geoip == NULL)
return NULL;
GeoIP_set_charset (geoip, GEOIP_CHARSET_UTF8);
LOG_DEBUG (("Opened legacy GeoIP database: %s\n", db));
return geoip;
}
static int
set_geoip_db_by_type (GeoIP *geoip, GO_GEOIP_DB type) {
unsigned char rec = GeoIP_database_edition (geoip);
switch (rec) {
case GEOIP_ASNUM_EDITION:
if (type != TYPE_ASN)
break;
geo_location_data = geoip;
return 0;
case GEOIP_COUNTRY_EDITION:
case GEOIP_COUNTRY_EDITION_V6:
if (type != TYPE_COUNTRY && type != TYPE_CITY)
break;
geo_location_data = geoip;
return 0;
case GEOIP_CITY_EDITION_REV0:
case GEOIP_CITY_EDITION_REV1:
case GEOIP_CITY_EDITION_REV0_V6:
case GEOIP_CITY_EDITION_REV1_V6:
if (type != TYPE_CITY)
break;
geo_location_data = geoip;
return 0;
}
return 1;
}
static int
set_conf_by_type (GeoIP *geoip) {
unsigned char rec = GeoIP_database_edition (geoip);
switch (rec) {
case GEOIP_ASNUM_EDITION:
conf.has_geoasn = 1;
return 0;
case GEOIP_COUNTRY_EDITION:
case GEOIP_COUNTRY_EDITION_V6:
conf.has_geocountry = 1;
return 0;
case GEOIP_CITY_EDITION_REV0:
case GEOIP_CITY_EDITION_REV1:
case GEOIP_CITY_EDITION_REV0_V6:
case GEOIP_CITY_EDITION_REV1_V6:
conf.has_geocountry = conf.has_geocity = 1;
return 0;
}
return 1;
}
/* Look up for a database on our array and set it as our current handlers.
* Note: this is not ideal, for now. However, legacy will go out of support at some point.
*
* On error or if no entry is found, 1 is returned.
* On success, GeoIP struct is set as our handler and 0 is returned. */
static int
set_geoip_db (GO_GEOIP_DB type) {
int idx = 0;
if (!is_geoip_resource ())
return 1;
/* in memory legacy DB */
if (legacy_db && geo_location_data)
return 0;
for (idx = 0; idx < db_cnt; idx++) {
if (set_geoip_db_by_type (geoips[idx], type) == 0)
return 0;
}
return 1;
}
static void
set_geoip (const char *db) {
GeoIP **new_geoips = NULL, *geoip = NULL;
if (db == NULL || *db == '\0')
return;
if (!(geoip = geoip_open_db (db)))
FATAL ("Unable to open GeoIP database %s\n", db);
db_cnt++;
new_geoips = realloc (geoips, sizeof (*geoips) * db_cnt);
if (new_geoips == NULL)
FATAL ("Unable to realloc GeoIP database %s\n", db);
geoips = new_geoips;
geoips[db_cnt - 1] = geoip;
set_conf_by_type (geoip);
}
/* Set up and open GeoIP database */
void
init_geoip (void) {
int i;
for (i = 0; i < conf.geoip_db_idx; ++i)
set_geoip (conf.geoip_databases[i]);
/* fall back to legacy GeoIP database */
if (!conf.geoip_db_idx) {
geo_location_data = GeoIP_new (conf.geo_db);
legacy_db = 1;
}
}
static char ip4to6_out_buffer[17];
static char *
ip4to6 (const char *ipv4) {
unsigned int b[4];
int n = sscanf (ipv4, "%u.%u.%u.%u", b, b + 1, b + 2, b + 3);
if (n == 4) {
snprintf (ip4to6_out_buffer, sizeof (ip4to6_out_buffer), "::ffff:%02x%02x:%02x%02x", b[0],
b[1], b[2], b[3]);
return ip4to6_out_buffer;
}
return NULL;
}
/* Get continent name concatenated with code.
*
* If continent not found, "Unknown" is returned.
* On success, the continent code & name is returned . */
static const char *
get_continent_name_and_code (const char *continentid) {
if (memcmp (continentid, "NA", 2) == 0)
return "NA North America";
else if (memcmp (continentid, "OC", 2) == 0)
return "OC Oceania";
else if (memcmp (continentid, "EU", 2) == 0)
return "EU Europe";
else if (memcmp (continentid, "SA", 2) == 0)
return "SA South America";
else if (memcmp (continentid, "AF", 2) == 0)
return "AF Africa";
else if (memcmp (continentid, "AN", 2) == 0)
return "AN Antarctica";
else if (memcmp (continentid, "AS", 2) == 0)
return "AS Asia";
else
return "-- Unknown";
}
/* Compose a string with the country name and code and store it in the
* given buffer. */
static void
geoip_set_country (const char *country, const char *code, char *loc) {
if (country && code)
snprintf (loc, COUNTRY_LEN, "%s %s", code, country);
else
snprintf (loc, COUNTRY_LEN, "%s", "Unknown");
}
/* Compose a string with the city name and state/region and store it
* in the given buffer. */
static void
geoip_set_city (const char *city, const char *region, char *loc) {
snprintf (loc, CITY_LEN, "%s, %s", city ? city : "N/A City", region ? region : "N/A Region");
}
/* Compose a string with the continent name and store it in the given
* buffer. */
static void
geoip_set_continent (const char *continent, char *loc) {
if (continent)
snprintf (loc, CONTINENT_LEN, "%s", get_continent_name_and_code (continent));
else
snprintf (loc, CONTINENT_LEN, "%s", "Unknown");
}
/* Compose a string with the continent name and store it in the given
* buffer. */
static void
geoip_set_asn (const char *name, char *asn) {
if (name)
snprintf (asn, ASN_LEN, "%s", name);
else
snprintf (asn, ASN_LEN, "%s", "Unknown");
}
/* Get detailed information found in the GeoIP Database about the
* given IPv4 or IPv6.
*
* On error, NULL is returned
* On success, GeoIPRecord structure is returned */
static GeoIPRecord *
get_geoip_record (const char *addr, GTypeIP type_ip) {
GeoIPRecord *rec = NULL;
if (TYPE_IPV4 == type_ip)
rec = GeoIP_record_by_name (geo_location_data, addr);
else if (TYPE_IPV6 == type_ip)
rec = GeoIP_record_by_name_v6 (geo_location_data, addr);
return rec;
}
/* Set country data by record into the given `location` buffer based
* on the IP version. */
static void
geoip_set_country_by_record (const char *ip, char *location, GTypeIP type_ip) {
GeoIPRecord *rec = NULL;
const char *country = NULL, *code = NULL, *addr = ip;
if (geo_location_data == NULL)
return;
/* Custom GeoIP database */
if ((rec = get_geoip_record (addr, type_ip))) {
country = rec->country_name;
code = rec->country_code;
}
geoip_set_country (country, code, location);
if (rec != NULL) {
GeoIPRecord_delete (rec);
}
}
/* Get the GeoIP location id by name.
*
* On error, 0 is returned
* On success, the GeoIP location id is returned */
static int
geoip_get_geoid (const char *addr, GTypeIP type_ip) {
int geoid = 0;
if (TYPE_IPV4 == type_ip)
geoid = GeoIP_id_by_name (geo_location_data, addr);
else if (TYPE_IPV6 == type_ip)
geoid = GeoIP_id_by_name_v6 (geo_location_data, addr);
return geoid;
}
/* Get the country name by GeoIP location id.
*
* On error, NULL is returned
* On success, the country name is returned */
static const char *
geoip_get_country_by_geoid (const char *addr, GTypeIP type_ip) {
const char *country = NULL;
if (TYPE_IPV4 == type_ip)
country = GeoIP_country_name_by_name (geo_location_data, addr);
else if (TYPE_IPV6 == type_ip)
country = GeoIP_country_name_by_name_v6 (geo_location_data, addr);
return country;
}
/* Set country data by geoid into the given `location` buffer based on
* the IP version. */
static void
geoip_set_country_by_geoid (const char *ip, char *location, GTypeIP type_ip) {
const char *country = NULL, *code = NULL, *addr = ip;
int geoid = 0;
if (!is_geoip_resource ())
return;
if (!(country = geoip_get_country_by_geoid (addr, type_ip)))
goto out;
/* return two letter country code */
if (!(geoid = geoip_get_geoid (addr, type_ip)))
goto out;
code = GeoIP_code_by_id (geoid);
out:
geoip_set_country (country, code, location);
}
/* Set continent data by record into the given `location` buffer based
* on the IP version. */
static void
geoip_set_continent_by_record (const char *ip, char *location, GTypeIP type_ip) {
GeoIPRecord *rec = NULL;
const char *continent = NULL, *addr = ip;
if (geo_location_data == NULL)
return;
/* Custom GeoIP database */
if ((rec = get_geoip_record (addr, type_ip)))
continent = rec->continent_code;
geoip_set_continent (continent, location);
if (rec != NULL) {
GeoIPRecord_delete (rec);
}
}
/* Set continent data by geoid into the given `location` buffer based
* on the IP version. */
static void
geoip_set_continent_by_geoid (const char *ip, char *location, GTypeIP type_ip) {
const char *continent = NULL, *addr = ip;
int geoid = 0;
if (!is_geoip_resource ())
return;
if (!(geoid = geoip_get_geoid (addr, type_ip)))
goto out;
continent = GeoIP_continent_by_id (geoid);
out:
geoip_set_continent (continent, location);
}
/* Set city data by record into the given `location` buffer based on
* the IP version. */
static void
geoip_set_city_by_record (const char *ip, char *location, GTypeIP type_ip) {
GeoIPRecord *rec = NULL;
const char *city = NULL, *region = NULL, *addr = ip;
/* Custom GeoIP database */
if ((rec = get_geoip_record (addr, type_ip))) {
city = rec->city;
region = rec->region;
}
geoip_set_city (city, region, location);
if (rec != NULL) {
GeoIPRecord_delete (rec);
}
}
/* Set city data by geoid or record into the given `location` buffer
* based on the IP version and currently used database edition.
* It uses the custom GeoIP database - i.e., GeoLiteCity.dat */
static void
geoip_get_city (const char *ip, char *location, GTypeIP type_ip) {
unsigned char rec = 0;
if (geo_location_data == NULL)
return;
rec = GeoIP_database_edition (geo_location_data);
switch (rec) {
case GEOIP_CITY_EDITION_REV0:
case GEOIP_CITY_EDITION_REV1:
if (TYPE_IPV4 == type_ip)
geoip_set_city_by_record (ip, location, TYPE_IPV4);
else
geoip_set_city (NULL, NULL, location);
break;
case GEOIP_CITY_EDITION_REV0_V6:
case GEOIP_CITY_EDITION_REV1_V6:
if (TYPE_IPV6 == type_ip)
geoip_set_city_by_record (ip, location, TYPE_IPV6);
else {
char *ipv6 = ip4to6 (ip);
if (ipv6)
geoip_set_city_by_record (ipv6, location, TYPE_IPV6);
else
geoip_set_city (NULL, NULL, location);
}
break;
}
}
/* Set country data by geoid or record into the given `location` buffer
* based on the IP version and currently used database edition. */
void
geoip_get_country (const char *ip, char *location, GTypeIP type_ip) {
unsigned char rec = 0;
if (set_geoip_db (TYPE_COUNTRY) && set_geoip_db (TYPE_CITY)) {
geoip_set_country (NULL, NULL, location);
return;
}
rec = GeoIP_database_edition (geo_location_data);
switch (rec) {
case GEOIP_COUNTRY_EDITION:
if (TYPE_IPV4 == type_ip)
geoip_set_country_by_geoid (ip, location, TYPE_IPV4);
else
geoip_set_country (NULL, NULL, location);
break;
case GEOIP_COUNTRY_EDITION_V6:
if (TYPE_IPV6 == type_ip)
geoip_set_country_by_geoid (ip, location, TYPE_IPV6);
else {
char *ipv6 = ip4to6 (ip);
if (ipv6)
geoip_set_country_by_geoid (ipv6, location, TYPE_IPV6);
else
geoip_set_country (NULL, NULL, location);
}
break;
case GEOIP_CITY_EDITION_REV0:
case GEOIP_CITY_EDITION_REV1:
if (TYPE_IPV4 == type_ip)
geoip_set_country_by_record (ip, location, TYPE_IPV4);
else
geoip_set_country (NULL, NULL, location);
break;
case GEOIP_CITY_EDITION_REV0_V6:
case GEOIP_CITY_EDITION_REV1_V6:
if (TYPE_IPV6 == type_ip)
geoip_set_country_by_record (ip, location, TYPE_IPV6);
else {
char *ipv6 = ip4to6 (ip);
if (ipv6)
geoip_set_country_by_record (ipv6, location, TYPE_IPV6);
else
geoip_set_country (NULL, NULL, location);
}
break;
}
}
/* Set continent data by geoid or record into the given `location` buffer
* based on the IP version and currently used database edition. */
void
geoip_get_continent (const char *ip, char *location, GTypeIP type_ip) {
unsigned char rec = 0;
if (set_geoip_db (TYPE_COUNTRY) && set_geoip_db (TYPE_CITY)) {
geoip_set_continent (NULL, location);
return;
}
rec = GeoIP_database_edition (geo_location_data);
switch (rec) {
case GEOIP_COUNTRY_EDITION:
if (TYPE_IPV4 == type_ip)
geoip_set_continent_by_geoid (ip, location, TYPE_IPV4);
else
geoip_set_continent (NULL, location);
break;
case GEOIP_COUNTRY_EDITION_V6:
if (TYPE_IPV6 == type_ip)
geoip_set_continent_by_geoid (ip, location, TYPE_IPV6);
else {
char *ipv6 = ip4to6 (ip);
if (ipv6)
geoip_set_continent_by_geoid (ipv6, location, TYPE_IPV6);
else
geoip_set_continent (NULL, location);
}
break;
case GEOIP_CITY_EDITION_REV0:
case GEOIP_CITY_EDITION_REV1:
if (TYPE_IPV4 == type_ip)
geoip_set_continent_by_record (ip, location, TYPE_IPV4);
else
geoip_set_continent (NULL, location);
break;
case GEOIP_CITY_EDITION_REV0_V6:
case GEOIP_CITY_EDITION_REV1_V6:
if (TYPE_IPV6 == type_ip)
geoip_set_continent_by_record (ip, location, TYPE_IPV6);
else {
char *ipv6 = ip4to6 (ip);
if (ipv6)
geoip_set_continent_by_record (ipv6, location, TYPE_IPV6);
else
geoip_set_continent (NULL, location);
}
break;
}
}
void
geoip_asn (char *host, char *asn) {
char *name = NULL;
if (legacy_db || set_geoip_db (TYPE_ASN)) {
geoip_set_asn (NULL, asn);
return;
}
/* Custom GeoIP database */
name = GeoIP_org_by_name (geo_location_data, (const char *) host);
geoip_set_asn (name, asn);
free (name);
}
/* Entry point to set GeoIP location into the corresponding buffers,
* (continent, country, city).
*
* On error, 1 is returned
* On success, buffers are set and 0 is returned */
int
set_geolocation (char *host, char *continent, char *country, char *city, GO_UNUSED char *asn) {
int type_ip = 0;
if (!is_geoip_resource ())
return 1;
if (invalid_ipaddr (host, &type_ip))
return 1;
/* set ASN data */
geoip_asn (host, asn);
/* set Country/City data */
if (set_geoip_db (TYPE_COUNTRY) == 0 || set_geoip_db (TYPE_CITY) == 0) {
geoip_get_country (host, country, type_ip);
geoip_get_continent (host, continent, type_ip);
}
if (set_geoip_db (TYPE_CITY) == 0)
geoip_get_city (host, city, type_ip);
return 0;
}
src/geoip1.h
+67
View File
@@ -0,0 +1,67 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#ifndef GEOIP_H_INCLUDED
#define GEOIP_H_INCLUDED
#include "commons.h"
#define CITY_LEN 47 + 1 /* max string length for a city */
#define CONTINENT_LEN 47 + 1 /* max string length for a country */
#define COUNTRY_LEN 48 + 3 /* Country + two-letter Code */
#define ASN_LEN 64 + 6 /* ASN + 5 digit/16-bit number/code */
/* Type of IP */
typedef enum {
TYPE_COUNTRY,
TYPE_CITY,
TYPE_ASN
} GO_GEOIP_DB;
typedef struct GLocation_ {
char city[CITY_LEN];
char continent[CONTINENT_LEN];
int hits;
} GLocation;
int is_geoip_resource (void);
int set_geolocation (char *host, char *continent, char *country, char *city, char *asn);
void geoip_asn (char *host, char *asn);
void geoip_free (void);
void geoip_get_continent (const char *ip, char *location, GTypeIP type_ip);
void geoip_get_country (const char *ip, char *location, GTypeIP type_ip);
void init_geoip (void);
#endif // for #ifndef GEOIP_H
void geoip_get_latlng (const char *host, double *lat, double *lng);
src/geoip2.c
+433
View File
@@ -0,0 +1,433 @@
/**
* geoip2.c -- implementation of GeoIP2
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2020 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#ifdef HAVE_LIBMAXMINDDB
#include <maxminddb.h>
#endif
#include "geoip1.h"
#include "error.h"
#include "labels.h"
#include "util.h"
#include "xmalloc.h"
/* should be reused across lookups */
static int geoip_asn_type = 0;
static int geoip_city_type = 0;
static int geoip_country_type = 0;
static int mmdb_cnt = 0;
static MMDB_s *mmdbs = NULL;
/* Determine if we have a valid geoip resource.
*
* If the geoip resource is NULL, 0 is returned.
* If the geoip resource is valid and malloc'd, 1 is returned. */
int
is_geoip_resource (void) {
return mmdbs && mmdb_cnt;
}
/* Free up GeoIP resources */
void
geoip_free (void) {
int idx = 0;
if (!is_geoip_resource ())
return;
for (idx = 0; idx < mmdb_cnt; idx++)
MMDB_close (&mmdbs[idx]);
free (mmdbs);
mmdbs = NULL;
}
static void
set_geoip (const char *db) {
int status = 0;
MMDB_s *new_mmdbs = NULL, mmdb;
if (db == NULL || *db == '\0')
return;
if ((status = MMDB_open (db, MMDB_MODE_MMAP, &mmdb)) != MMDB_SUCCESS)
FATAL ("Unable to open GeoIP2 database %s: %s\n", db, MMDB_strerror (status));
mmdb_cnt++;
new_mmdbs = realloc (mmdbs, sizeof (*mmdbs) * mmdb_cnt);
if (new_mmdbs == NULL)
FATAL ("Unable to realloc GeoIP2 database %s\n", db);
mmdbs = new_mmdbs;
mmdbs[mmdb_cnt - 1] = mmdb;
if (strstr (mmdb.metadata.database_type, "-City") != NULL)
conf.has_geocountry = conf.has_geocity = geoip_country_type = geoip_city_type = 1;
if (strstr (mmdb.metadata.database_type, "-ASN") != NULL)
conf.has_geoasn = geoip_asn_type = 1;
if (strstr (mmdb.metadata.database_type, "-Country") != NULL)
conf.has_geocountry = geoip_country_type = 1;
}
/* Open the given GeoIP2 database.
*
* On error, it aborts.
* On success, a new geolocation structure is set. */
void
init_geoip (void) {
int i;
for (i = 0; i < conf.geoip_db_idx; ++i)
set_geoip (conf.geoip_databases[i]);
}
/* Look up an IP address that is passed in as a null-terminated string.
*
* On error, it aborts.
* If no entry is found, 1 is returned.
* On success, MMDB_lookup_result_s struct is set and 0 is returned. */
static int
geoip_lookup (MMDB_lookup_result_s *res, const char *ip, int is_asn) {
int gai_err, mmdb_err, idx = 0;
MMDB_s *mmdb = NULL;
for (idx = 0; idx < mmdb_cnt; idx++) {
if (is_asn && (!strstr (mmdbs[idx].metadata.database_type, "ASN")))
continue;
if (!is_asn && (strstr (mmdbs[idx].metadata.database_type, "ASN")))
continue;
mmdb = &mmdbs[idx];
*res = MMDB_lookup_string (mmdb, ip, &gai_err, &mmdb_err);
if (0 != gai_err)
return 1;
if (MMDB_SUCCESS != mmdb_err)
FATAL ("Error from libmaxminddb: %s\n", MMDB_strerror (mmdb_err));
if (!(*res).found_entry)
return 1;
break;
}
return 0;
}
/* Get continent name concatenated with code.
*
* If continent not found, "Unknown" is returned.
* On success, the continent code & name is returned . */
static const char *
get_continent_name_and_code (const char *continentid) {
if (memcmp (continentid, "NA", 2) == 0)
return "NA North America";
else if (memcmp (continentid, "OC", 2) == 0)
return "OC Oceania";
else if (memcmp (continentid, "EU", 2) == 0)
return "EU Europe";
else if (memcmp (continentid, "SA", 2) == 0)
return "SA South America";
else if (memcmp (continentid, "AF", 2) == 0)
return "AF Africa";
else if (memcmp (continentid, "AN", 2) == 0)
return "AN Antarctica";
else if (memcmp (continentid, "AS", 2) == 0)
return "AS Asia";
else
return "-- Unknown";
}
/* Compose a string with the country name and code and store it in the
* given buffer. */
static void
geoip_set_country (const char *country, const char *code, char *loc) {
if (country && code)
snprintf (loc, COUNTRY_LEN, "%s %s", code, country);
else
snprintf (loc, COUNTRY_LEN, "%s", "Unknown");
}
/* Compose a string with the ASN name and code and store it in the
* given buffer. */
static void
geoip_set_asn (MMDB_entry_data_s name, MMDB_entry_data_s code, char *asn, int status) {
if (status == 0)
snprintf (asn, ASN_LEN, "%05u: %.*s", code.uint32, name.data_size, name.utf8_string);
else
snprintf (asn, ASN_LEN, "%s", "00000: Unknown");
}
/* Compose a string with the city name and state/region and store it
* in the given buffer. */
static void
geoip_set_city (const char *city, char *loc) {
snprintf (loc, CITY_LEN, "%s", city ? city : "N/A City");
}
/* Compose a string with the continent name and store it in the given
* buffer. */
static void
geoip_set_continent (const char *continent, char *loc) {
if (continent)
snprintf (loc, CONTINENT_LEN, "%s", get_continent_name_and_code (continent));
else
snprintf (loc, CONTINENT_LEN, "%s", "Unknown");
}
/* A wrapper to fetch the looked up result set.
*
* On error, it aborts.
* If no data is found, NULL is returned.
* On success, the fetched value is returned. */
static char *
get_value (MMDB_lookup_result_s res, ...) {
MMDB_entry_data_s entry_data;
char *value = NULL;
int status = 0;
va_list keys;
va_start (keys, res);
status = MMDB_vget_value (&res.entry, &entry_data, keys);
va_end (keys);
if (status != MMDB_SUCCESS)
return NULL;
if (!entry_data.has_data)
return NULL;
if (entry_data.type != MMDB_DATA_TYPE_UTF8_STRING)
FATAL ("Invalid data UTF8 GeoIP2 data %d:\n", entry_data.type);
if ((value = strndup (entry_data.utf8_string, entry_data.data_size)) == NULL)
FATAL ("Unable to allocate buffer %s: ", strerror (errno));
return value;
}
/* A wrapper to fetch the looked up result and set the city and region.
*
* If no data is found, NULL is set.
* On success, the fetched value is set. */
static void
geoip_query_city (MMDB_lookup_result_s res, char *location) {
char *city = NULL, *region = NULL;
if (res.found_entry) {
city = get_value (res, "city", "names", DOC_LANG, NULL);
region = get_value (res, "subdivisions", "0", "names", DOC_LANG, NULL);
if (!city) {
city = get_value (res, "city", "names", "en", NULL);
}
if (!region) {
region = get_value (res, "subdivisions", "0", "names", "en", NULL);
}
}
geoip_set_city (city, location);
free (city);
free (region);
}
/* A wrapper to fetch the looked up result and set the country and code.
*
* If no data is found, NULL is set.
* On success, the fetched value is set. */
static void
geoip_query_country (MMDB_lookup_result_s res, char *location) {
char *country = NULL, *code = NULL;
if (res.found_entry) {
country = get_value (res, "country", "names", DOC_LANG, NULL);
code = get_value (res, "country", "iso_code", NULL);
if (!country) {
country = get_value (res, "country", "names", "en", NULL);
}
}
geoip_set_country (country, code, location);
free (code);
free (country);
}
/* Sets the value for the given array of strings as the lookup path.
*
* On error or not found, 1 is returned.
* On success, 0 is returned and the entry data is set. */
static int
geoip_query_asn_code (MMDB_lookup_result_s res, MMDB_entry_data_s *code) {
int status;
const char *key[] = { "autonomous_system_number", NULL };
if (res.found_entry) {
status = MMDB_aget_value (&res.entry, code, key);
if (status != MMDB_SUCCESS || !code->has_data)
return 1;
}
return 0;
}
/* Sets the value for the given array of strings as the lookup path.
*
* On error or not found, 1 is returned.
* On success, 0 is returned and the entry data is set. */
static int
geoip_query_asn_name (MMDB_lookup_result_s res, MMDB_entry_data_s *name) {
int status;
const char *key[] = { "autonomous_system_organization", NULL };
if (res.found_entry) {
status = MMDB_aget_value (&res.entry, name, key);
if (status != MMDB_SUCCESS || !name->has_data)
return 1;
}
return 0;
}
/* A wrapper to fetch the looked up result and set the ASN organization & code.
*
* If no data is found, "Unknown" is set.
* On success, the fetched value is set. */
void
geoip_asn (char *host, char *asn) {
MMDB_lookup_result_s res = { 0 };
MMDB_entry_data_s name = { 0 };
MMDB_entry_data_s code = { 0 };
int status = 1;
geoip_lookup (&res, host, 1);
if (!res.found_entry)
goto out;
if ((status &= geoip_query_asn_name (res, &name)))
goto out;
if ((status |= geoip_query_asn_code (res, &code)))
goto out;
out:
geoip_set_asn (name, code, asn, status);
}
/* A wrapper to fetch the looked up result and set the continent code.
*
* If no data is found, NULL is set.
* On success, the fetched value is set. */
static void
geoip_query_continent (MMDB_lookup_result_s res, char *location) {
char *code = NULL;
if (res.found_entry)
code = get_value (res, "continent", "code", NULL);
geoip_set_continent (code, location);
free (code);
}
/* Set country data by record into the given `location` buffer */
void
geoip_get_country (const char *ip, char *location, GO_UNUSED GTypeIP type_ip) {
MMDB_lookup_result_s res = { 0 };
geoip_lookup (&res, ip, 0);
geoip_query_country (res, location);
}
/* A wrapper to fetch the looked up result and set the continent. */
void
geoip_get_continent (const char *ip, char *location, GO_UNUSED GTypeIP type_ip) {
MMDB_lookup_result_s res = { 0 };
geoip_lookup (&res, ip, 0);
geoip_query_continent (res, location);
}
/* Extract lat/lng coordinates from mmdb for the given host. */
static void
geoip_get_coordinates (const char *host, double *lat, double *lng) {
MMDB_lookup_result_s res = { 0 };
MMDB_entry_data_s entry_data;
int status;
const char *lat_keys[] = {"location", "latitude", NULL};
const char *lng_keys[] = {"location", "longitude", NULL};
*lat = *lng = 0.0;
geoip_lookup (&res, host, 0);
if (!res.found_entry)
return;
status = MMDB_aget_value (&res.entry, &entry_data, lat_keys);
if (status == MMDB_SUCCESS && entry_data.has_data && entry_data.type == MMDB_DATA_TYPE_DOUBLE)
*lat = entry_data.double_value;
status = MMDB_aget_value (&res.entry, &entry_data, lng_keys);
if (status == MMDB_SUCCESS && entry_data.has_data && entry_data.type == MMDB_DATA_TYPE_DOUBLE)
*lng = entry_data.double_value;
}
/* Get lat/lng for a given IP (public wrapper). */
void
geoip_get_latlng (const char *host, double *lat, double *lng) {
geoip_get_coordinates (host, lat, lng);
}
/* Entry point to set GeoIP location into the corresponding buffers,
* (continent, country, city).
*
* On error, 1 is returned
* On success, buffers are set and 0 is returned */
int
set_geolocation (char *host, char *continent, char *country, char *city, char *asn) {
MMDB_lookup_result_s res = { 0 };
if (!is_geoip_resource ())
return 1;
/* set ASN data */
if (geoip_asn_type)
geoip_asn (host, asn);
if (!geoip_city_type && !geoip_country_type)
return 0;
/* set Country/City data */
geoip_lookup (&res, host, 0);
geoip_query_country (res, country);
geoip_query_continent (res, continent);
if (geoip_city_type)
geoip_query_city (res, city);
return 0;
}
src/gholder.c
+897
View File
@@ -0,0 +1,897 @@
/**
* gholder.c -- data structure to hold raw data
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include <pthread.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include "gholder.h"
#include "error.h"
#include "gdns.h"
#include "gkhash.h"
#include "gstorage.h"
#include "util.h"
#include "xmalloc.h"
#ifdef HAVE_GEOLOCATION
#include "geoip1.h"
#endif
typedef struct GPanel_ {
GModule module;
void (*insert) (GRawDataItem item, GHolder * h, datatype type, const struct GPanel_ *);
void (*holder_callback) (GHolder * h);
} GPanel;
/* *INDENT-OFF* */
static void add_data_to_holder (GRawDataItem item, GHolder * h, datatype type, const GPanel * panel);
static void add_host_to_holder (GRawDataItem item, GHolder * h, datatype type, const GPanel * panel);
static void add_root_to_holder (GRawDataItem item, GHolder * h, datatype type, const GPanel * panel);
static void add_host_child_to_holder (GHolder * h);
#ifdef HAVE_GEOLOCATION
static void add_geo_to_holder (GRawDataItem item, GHolder * h, datatype type, const GPanel * panel);
#endif
static const GPanel paneling[] = {
{VISITORS , add_data_to_holder , NULL},
{REQUESTS , add_data_to_holder , NULL},
{REQUESTS_STATIC , add_data_to_holder , NULL},
{NOT_FOUND , add_data_to_holder , NULL},
{HOSTS , add_host_to_holder , add_host_child_to_holder} ,
{OS , add_root_to_holder , NULL},
{BROWSERS , add_root_to_holder , NULL},
{VISIT_TIMES , add_data_to_holder , NULL},
{VIRTUAL_HOSTS , add_data_to_holder , NULL},
{REFERRERS , add_data_to_holder , NULL},
{REFERRING_SITES , add_data_to_holder , NULL},
{KEYPHRASES , add_data_to_holder , NULL},
{STATUS_CODES , add_root_to_holder , NULL},
{REMOTE_USER , add_data_to_holder , NULL},
{CACHE_STATUS , add_data_to_holder , NULL},
#ifdef HAVE_GEOLOCATION
{GEO_LOCATION , add_geo_to_holder , NULL},
{ASN , add_data_to_holder , NULL} ,
#endif
{MIME_TYPE , add_root_to_holder , NULL} ,
{TLS_TYPE , add_root_to_holder , NULL} ,
};
/* *INDENT-ON* */
/* Get a panel from the GPanel structure given a module.
*
* On error, or if not found, NULL is returned.
* On success, the panel value is returned. */
static const GPanel *
panel_lookup (GModule module) {
int i, num_panels = ARRAY_SIZE (paneling);
for (i = 0; i < num_panels; i++) {
if (paneling[i].module == module)
return &paneling[i];
}
return NULL;
}
/* Allocate memory for a new GHolder instance.
*
* On success, the newly allocated GHolder is returned . */
GHolder *
new_gholder (uint32_t size) {
GHolder *holder = xmalloc (size * sizeof (GHolder));
memset (holder, 0, size * sizeof *holder);
return holder;
}
/* Allocate memory for a new GHolderItem instance.
*
* On success, the newly allocated GHolderItem is returned . */
static GHolderItem *
new_gholder_item (uint32_t size) {
GHolderItem *item = xcalloc (size, sizeof (GHolderItem));
return item;
}
/* Allocate memory for a new double linked-list GSubList instance.
*
* On success, the newly allocated GSubList is returned . */
static GSubList *
new_gsublist (void) {
GSubList *sub_list = xmalloc (sizeof (GSubList));
sub_list->head = NULL;
sub_list->tail = NULL;
sub_list->size = 0;
return sub_list;
}
/* Allocate memory for a new double linked-list GSubItem node.
*
* On success, the newly allocated GSubItem is returned . */
static GSubItem *
new_gsubitem (GModule module, GMetrics *nmetrics) {
GSubItem *sub_item = xmalloc (sizeof (GSubItem));
sub_item->metrics = nmetrics;
sub_item->module = module;
sub_item->sub_list = NULL;
sub_item->prev = NULL;
sub_item->next = NULL;
return sub_item;
}
/* Add an item to the end of a given sub list. */
static void
add_sub_item_back (GSubList *sub_list, GModule module, GMetrics *nmetrics) {
GSubItem *sub_item = new_gsubitem (module, nmetrics);
if (sub_list->tail) {
sub_list->tail->next = sub_item;
sub_item->prev = sub_list->tail;
sub_list->tail = sub_item;
} else {
sub_list->head = sub_item;
sub_list->tail = sub_item;
}
sub_list->size++;
}
/* Delete the entire given sub list. */
static void
delete_sub_list (GSubList *sub_list) {
GSubItem *item = NULL;
GSubItem *next = NULL;
if (sub_list->size == 0)
goto clear;
for (item = sub_list->head; item; item = next) {
next = item->next;
if (item->sub_list != NULL)
delete_sub_list (item->sub_list);
free (item->metrics->data);
free (item->metrics);
free (item);
}
clear:
sub_list->head = NULL;
sub_list->size = 0;
free (sub_list);
}
/* Free malloc'd holder fields. */
static void
free_holder_data (GHolderItem item) {
if (item.sub_list != NULL)
delete_sub_list (item.sub_list);
free_gmetrics (item.metrics);
}
/* Free all memory allocated in holder for a given module. */
void
free_holder_by_module (GHolder **holder, GModule module) {
uint32_t j;
if ((*holder) == NULL)
return;
for (j = 0; j < (*holder)[module].idx; j++) {
free_holder_data ((*holder)[module].items[j]);
}
free ((*holder)[module].items);
(*holder)[module].holder_size = 0;
(*holder)[module].idx = 0;
(*holder)[module].sub_items_size = 0;
}
/* Free all memory allocated in holder for all modules. */
void
free_holder (GHolder **holder) {
GModule module;
uint32_t j;
size_t idx = 0;
if ((*holder) == NULL)
return;
FOREACH_MODULE (idx, module_list) {
module = module_list[idx];
for (j = 0; j < (*holder)[module].idx; j++) {
free_holder_data ((*holder)[module].items[j]);
}
free ((*holder)[module].items);
}
free (*holder);
(*holder) = NULL;
}
/* Iterate over holder and get the key index.
*
* If the key does not exist, -1 is returned.
* On success, the key in holder is returned . */
static int
get_item_idx_in_holder (GHolder *holder, const char *k) {
uint32_t i;
if (holder == NULL)
return KEY_NOT_FOUND;
if (holder->idx == 0)
return KEY_NOT_FOUND;
if (k == NULL || *k == '\0')
return KEY_NOT_FOUND;
for (i = 0; i < holder->idx; i++) {
if (strcmp (k, holder->items[i].metrics->data) == 0)
return i;
}
return KEY_NOT_FOUND;
}
/* Sort a single GSubList in place: copy to array, sort, rebuild list.
* Preserves each sub-item's own sub_list for recursive nesting. */
static void
sort_single_sub_list (GSubList **sl_ptr, GModule module, GSort sort, int max_choices_sub) {
GHolderItem *arr;
GSubItem *iter;
GSubList *sub_list = *sl_ptr;
int j, k, limit;
if (sub_list == NULL || sub_list->size == 0)
return;
arr = new_gholder_item (sub_list->size);
/* copy items from the linked-list into an array, preserving sub_list */
for (j = 0, iter = sub_list->head; iter; iter = iter->next, j++) {
arr[j].metrics = new_gmetrics ();
arr[j].metrics->nbw = iter->metrics->nbw;
arr[j].metrics->data = xstrdup (iter->metrics->data);
arr[j].metrics->hits = iter->metrics->hits;
arr[j].metrics->id = iter->metrics->id;
arr[j].metrics->visitors = iter->metrics->visitors;
arr[j].sub_list = iter->sub_list;
iter->sub_list = NULL; /* prevent double-free */
if (conf.serve_usecs) {
arr[j].metrics->avgts.nts = iter->metrics->avgts.nts;
arr[j].metrics->cumts.nts = iter->metrics->cumts.nts;
arr[j].metrics->maxts.nts = iter->metrics->maxts.nts;
}
}
sort_holder_items (arr, j, sort);
delete_sub_list (sub_list);
/* Only rebuild up to max_choices_sub items */
limit = (j < max_choices_sub) ? j : max_choices_sub;
sub_list = new_gsublist ();
for (k = 0; k < limit; k++) {
if (k > 0)
sub_list = *sl_ptr;
add_sub_item_back (sub_list, module, arr[k].metrics);
sub_list->tail->sub_list = arr[k].sub_list;
*sl_ptr = sub_list;
sub_list = NULL;
}
/* Free items beyond max_choices_sub */
for (k = limit; k < j; k++) {
if (arr[k].sub_list != NULL)
delete_sub_list (arr[k].sub_list);
free (arr[k].metrics->data);
free (arr[k].metrics);
}
free (arr);
if (sub_list) {
delete_sub_list (sub_list);
sub_list = NULL;
}
/* recursively sort nested sub-lists */
sub_list = *sl_ptr;
if (sub_list != NULL) {
for (iter = sub_list->head; iter; iter = iter->next) {
if (iter->sub_list != NULL)
sort_single_sub_list (&iter->sub_list, module, sort, max_choices_sub);
}
}
}
/* Copy linked-list items to an array, sort, and move them back to the
* list. Should be faster than sorting the list */
static void
sort_sub_list (GHolder *h, GSort sort) {
uint32_t i;
/* iterate over root-level nodes */
for (i = 0; i < h->idx; i++) {
sort_single_sub_list (&h->items[i].sub_list, h->module, sort, h->max_choices_sub);
}
}
/* Set the data metric field for the host panel.
*
* On success, the data field/metric is set. */
static int
set_host_child_metrics (char *data, uint8_t id, GMetrics **nmetrics) {
GMetrics *metrics;
metrics = new_gmetrics ();
metrics->data = xstrdup (data);
metrics->id = id;
*nmetrics = metrics;
return 0;
}
/* Set host panel data, including sub items.
*
* On success, the host panel data is set. */
static void
set_host_sub_list (GHolder *h, GSubList *sub_list) {
GMetrics *nmetrics;
#ifdef HAVE_GEOLOCATION
char city[CITY_LEN] = "";
char continent[CONTINENT_LEN] = "";
char country[ASN_LEN] = "";
char asn[ASN_LEN] = "";
#endif
char *host = h->items[h->idx].metrics->data, *hostname = NULL;
#ifdef HAVE_GEOLOCATION
/* add geolocation child nodes */
set_geolocation (host, continent, country, city, asn);
/* country */
if (country[0] != '\0' && sub_list->size < h->max_choices_sub) {
set_host_child_metrics (country, MTRC_ID_COUNTRY, &nmetrics);
add_sub_item_back (sub_list, h->module, nmetrics);
h->items[h->idx].sub_list = sub_list;
h->sub_items_size++;
/* flag only */
conf.has_geocountry = 1;
}
/* city */
if (city[0] != '\0' && sub_list->size < h->max_choices_sub) {
double lat = 0.0, lng = 0.0;
geoip_get_latlng (host, &lat, &lng);
set_host_child_metrics (city, MTRC_ID_CITY, &nmetrics);
nmetrics->lat = lat;
nmetrics->lng = lng;
add_sub_item_back (sub_list, h->module, nmetrics);
h->items[h->idx].sub_list = sub_list;
h->sub_items_size++;
/* flag only */
conf.has_geocity = 1;
}
/* ASN */
if (asn[0] != '\0' && sub_list->size < h->max_choices_sub) {
set_host_child_metrics (asn, MTRC_ID_ASN, &nmetrics);
add_sub_item_back (sub_list, h->module, nmetrics);
h->items[h->idx].sub_list = sub_list;
h->sub_items_size++;
/* flag only */
conf.has_geoasn = 1;
}
#endif
/* hostname */
if (conf.enable_html_resolver && conf.output_stdout && !conf.no_ip_validation &&
!conf.real_time_html && sub_list->size < h->max_choices_sub) {
hostname = reverse_ip (host);
set_host_child_metrics (hostname, MTRC_ID_HOSTNAME, &nmetrics);
add_sub_item_back (sub_list, h->module, nmetrics);
h->items[h->idx].sub_list = sub_list;
h->sub_items_size++;
free (hostname);
}
}
/* Set host panel data, including sub items.
*
* On success, the host panel data is set. */
static void
add_host_child_to_holder (GHolder *h) {
GMetrics *nmetrics;
GSubList *sub_list = new_gsublist ();
char *ip = h->items[h->idx].metrics->data;
char *hostname = NULL;
uint32_t n = h->sub_items_size;
/* add child nodes */
set_host_sub_list (h, sub_list);
pthread_mutex_lock (&gdns_thread.mutex);
hostname = ht_get_hostname (ip);
pthread_mutex_unlock (&gdns_thread.mutex);
/* determine if we have the IP's hostname */
if (!hostname) {
dns_resolver (ip);
} else if (hostname && sub_list->size < h->max_choices_sub) {
set_host_child_metrics (hostname, MTRC_ID_HOSTNAME, &nmetrics);
add_sub_item_back (sub_list, h->module, nmetrics);
h->items[h->idx].sub_list = sub_list;
h->sub_items_size++;
free (hostname);
} else if (hostname) {
free (hostname);
}
/* did not add any items */
if (n == h->sub_items_size)
free (sub_list);
}
/* Given a GRawDataType, set the data and hits value.
*
* On error, no values are set and 1 is returned.
* On success, the data and hits values are set and 0 is returned. */
static int
map_data (GModule module, GRawDataItem item, datatype type, char **data, uint32_t *hits) {
switch (type) {
case U32:
if (!(*data = ht_get_datamap (module, item.nkey)))
return 1;
if (!item.hits)
return 1;
*hits = item.hits;
break;
case STR:
if (!(*hits = ht_get_hits (module, item.nkey)))
return 1;
*data = xstrdup (item.data);
break;
}
return 0;
}
/* Given a data item, store it into a holder structure. */
static void
set_single_metrics (GRawDataItem item, GHolder *h, char *data, uint32_t hits) {
uint32_t visitors = 0;
uint64_t bw = 0, cumts = 0, maxts = 0;
bw = ht_get_bw (h->module, item.nkey);
cumts = ht_get_cumts (h->module, item.nkey);
maxts = ht_get_maxts (h->module, item.nkey);
visitors = ht_get_visitors (h->module, item.nkey);
h->items[h->idx].metrics = new_gmetrics ();
h->items[h->idx].metrics->hits = hits;
h->items[h->idx].metrics->data = data;
h->items[h->idx].metrics->visitors = visitors;
h->items[h->idx].metrics->nbw = bw;
h->items[h->idx].metrics->avgts.nts = cumts / hits;
h->items[h->idx].metrics->cumts.nts = cumts;
h->items[h->idx].metrics->maxts.nts = maxts;
if (bw && !conf.bandwidth)
conf.bandwidth = 1;
if (cumts && !conf.serve_usecs)
conf.serve_usecs = 1;
if (conf.append_method) {
h->items[h->idx].metrics->method = ht_get_method (h->module, item.nkey);
}
if (conf.append_protocol) {
h->items[h->idx].metrics->protocol = ht_get_protocol (h->module, item.nkey);
}
}
/* Set all panel data. This will set data for panels that do not
* contain sub items. A function pointer is used for post data set. */
static void
add_data_to_holder (GRawDataItem item, GHolder *h, datatype type, const GPanel *panel) {
char *data = NULL;
uint32_t hits = 0;
if (map_data (h->module, item, type, &data, &hits) == 1)
return;
set_single_metrics (item, h, data, hits);
if (panel->holder_callback)
panel->holder_callback (h);
h->idx++;
}
/* A wrapper to set a host item */
static void
set_host (GRawDataItem item, GHolder *h, const GPanel *panel, char *data, uint32_t hits) {
set_single_metrics (item, h, xstrdup (data), hits);
if (panel->holder_callback)
panel->holder_callback (h);
h->idx++;
}
/* Set all panel data. This will set data for panels that do not
* contain sub items. A function pointer is used for post data set. */
static void
add_host_to_holder (GRawDataItem item, GHolder *h, datatype type, const GPanel *panel) {
char buf4[INET_ADDRSTRLEN];
char buf6[INET6_ADDRSTRLEN];
char *data = NULL;
uint32_t hits = 0;
unsigned i;
struct in6_addr addr6, mask6, nwork6;
struct in_addr addr4, mask4, nwork4;
const char *arr4[] = { "255.255.255.0", "255.255.0.0", "255.0.0.0" };
const char *arr6[] = {
"ffff:ffff:ffff:ffff:0000:0000:0000:0000",
"ffff:ffff:ffff:0000:0000:0000:0000:0000",
"ffff:ffff:0000:0000:0000:0000:0000:0000"
};
const char *m4 = arr4[0];
const char *m6 = arr6[0];
if (map_data (h->module, item, type, &data, &hits) == 1)
return;
if (!conf.anonymize_ip) {
add_data_to_holder (item, h, type, panel);
free (data);
return;
}
if (conf.anonymize_level == ANONYMIZE_STRONG || conf.anonymize_level == ANONYMIZE_PEDANTIC) {
m4 = arr4[conf.anonymize_level - 1];
m6 = arr6[conf.anonymize_level - 1];
}
if (1 == inet_pton (AF_INET, data, &addr4)) {
if (1 == inet_pton (AF_INET, m4, &mask4)) {
memset (buf4, 0, sizeof *buf4);
nwork4.s_addr = addr4.s_addr & mask4.s_addr;
if (inet_ntop (AF_INET, &nwork4, buf4, INET_ADDRSTRLEN) != NULL) {
set_host (item, h, panel, buf4, hits);
free (data);
}
}
} else if (1 == inet_pton (AF_INET6, data, &addr6)) {
if (1 == inet_pton (AF_INET6, m6, &mask6)) {
memset (buf6, 0, sizeof *buf6);
for (i = 0; i < 16; i++) {
nwork6.s6_addr[i] = addr6.s6_addr[i] & mask6.s6_addr[i];
}
if (inet_ntop (AF_INET6, &nwork6, buf6, INET6_ADDRSTRLEN) != NULL) {
set_host (item, h, panel, buf6, hits);
free (data);
}
}
}
}
/* Set all root panel data. This will set the root nodes. */
static int
set_root_metrics (GRawDataItem item, GModule module, datatype type, GMetrics **nmetrics) {
GMetrics *metrics;
char *data = NULL;
uint64_t bw = 0, cumts = 0, maxts = 0;
uint32_t hits = 0, visitors = 0;
if (map_data (module, item, type, &data, &hits) == 1)
return 1;
bw = ht_get_bw (module, item.nkey);
cumts = ht_get_cumts (module, item.nkey);
maxts = ht_get_maxts (module, item.nkey);
visitors = ht_get_visitors (module, item.nkey);
metrics = new_gmetrics ();
metrics->avgts.nts = cumts / hits;
metrics->cumts.nts = cumts;
metrics->maxts.nts = maxts;
metrics->nbw = bw;
metrics->data = data;
metrics->hits = hits;
metrics->visitors = visitors;
*nmetrics = metrics;
return 0;
}
/* Set all root panel data, including sub list items. */
static void
add_root_to_holder (GRawDataItem item, GHolder *h, datatype type, GO_UNUSED const GPanel *panel) {
GSubList *sub_list;
GMetrics *metrics, *nmetrics;
char *root = NULL;
int root_idx = KEY_NOT_FOUND, idx = 0;
if (set_root_metrics (item, h->module, type, &nmetrics) == 1)
return;
if (!(root = ht_get_root (h->module, item.nkey))) {
free_gmetrics (nmetrics);
return;
}
/* add data as a child node into holder */
if (KEY_NOT_FOUND == (root_idx = get_item_idx_in_holder (h, root))) {
/* Check if we've reached max_choices for root items */
if (h->idx >= h->max_choices) {
free (root);
free_gmetrics (nmetrics);
return;
}
idx = h->idx;
sub_list = new_gsublist ();
metrics = new_gmetrics ();
h->items[idx].metrics = metrics;
h->items[idx].metrics->data = root;
h->idx++;
} else {
sub_list = h->items[root_idx].sub_list;
metrics = h->items[root_idx].metrics;
idx = root_idx;
free (root);
}
/* Accumulate metrics into parent regardless of whether we add the sub-item */
h->items[idx].metrics = metrics;
h->items[idx].metrics->cumts.nts += nmetrics->cumts.nts;
h->items[idx].metrics->nbw += nmetrics->nbw;
h->items[idx].metrics->hits += nmetrics->hits;
h->items[idx].metrics->visitors += nmetrics->visitors;
h->items[idx].metrics->avgts.nts = h->items[idx].metrics->cumts.nts / h->items[idx].metrics->hits;
if (nmetrics->maxts.nts > h->items[idx].metrics->maxts.nts)
h->items[idx].metrics->maxts.nts = nmetrics->maxts.nts;
/* Only add sub-item if we haven't reached max_choices_sub for this sub-list */
if (sub_list->size < h->max_choices_sub) {
add_sub_item_back (sub_list, h->module, nmetrics);
h->items[idx].sub_list = sub_list;
h->sub_items_size++;
} else {
free_gmetrics (nmetrics);
}
}
#ifdef HAVE_GEOLOCATION
/* Find a sub-item in a GSubList by its data string.
* Returns the matching GSubItem, or NULL if not found. */
static GSubItem *
find_sub_item_by_data (GSubList *sl, const char *data) {
GSubItem *iter;
if (sl == NULL || data == NULL)
return NULL;
for (iter = sl->head; iter; iter = iter->next) {
if (strcmp (iter->metrics->data, data) == 0)
return iter;
}
return NULL;
}
/* Build 3-level GEO_LOCATION hierarchy: Continent > Country > City.
* Falls back to add_root_to_holder (2-level) when has_geocity is false. */
static void
add_geo_to_holder (GRawDataItem item, GHolder *h, datatype type, GO_UNUSED const GPanel *panel) {
GMetrics *nmetrics;
GSubList *sub_list;
GSubItem *country_sub;
GMetrics *cont_metrics, *country_metrics;
char *root = NULL;
const char *continent = NULL;
int root_idx = KEY_NOT_FOUND, idx = 0;
if (!conf.has_geocity) {
add_root_to_holder (item, h, type, panel);
return;
}
/* city metrics from storage (city is the "data" key) */
if (set_root_metrics (item, h->module, type, &nmetrics) == 1)
return;
/* country is the "root" of the city in storage */
if (!(root = ht_get_root (h->module, item.nkey))) {
free_gmetrics (nmetrics);
return;
}
/* Extract continent from root if encoded as "continent|country".
* Falls back to country_continent_map for old persisted data. */
{
char *sep = strchr (root, 0x7c);
if (sep) {
*sep = 0x00;
continent = xstrdup (root); /* copy before freeing root */
char *new_root = xstrdup (sep + 1);
free (root);
root = new_root;
}
}
if (!continent)
continent = get_continent_for_country (root);
if (continent == NULL) {
/* fallback: use country as root directly (2-level) */
free (root);
add_root_to_holder (item, h, type, panel);
free_gmetrics (nmetrics);
return;
}
/* Find or create continent as root GHolderItem */
if (KEY_NOT_FOUND == (root_idx = get_item_idx_in_holder (h, continent))) {
/* Check if we've reached max_choices for root items (continents) */
if (h->idx >= h->max_choices) {
free (root);
free_gmetrics (nmetrics);
return;
}
idx = h->idx;
cont_metrics = new_gmetrics ();
cont_metrics->data = xstrdup (continent);
h->items[idx].metrics = cont_metrics;
h->items[idx].sub_list = new_gsublist ();
h->idx++;
} else {
idx = root_idx;
cont_metrics = h->items[idx].metrics;
}
/* Find or create country as sub-item under continent */
sub_list = h->items[idx].sub_list;
country_sub = find_sub_item_by_data (sub_list, root);
if (country_sub == NULL) {
/* Only add country if continent's sub-list hasn't reached max_choices_sub */
if (sub_list->size < h->max_choices_sub) {
country_metrics = new_gmetrics ();
country_metrics->data = xstrdup (root);
add_sub_item_back (sub_list, h->module, country_metrics);
country_sub = sub_list->tail;
country_sub->sub_list = new_gsublist ();
h->sub_items_size++;
} else {
/* Continent sub-list is full, accumulate to continent and skip country/city */
cont_metrics->hits += nmetrics->hits;
cont_metrics->visitors += nmetrics->visitors;
cont_metrics->nbw += nmetrics->nbw;
cont_metrics->cumts.nts += nmetrics->cumts.nts;
if (nmetrics->maxts.nts > cont_metrics->maxts.nts)
cont_metrics->maxts.nts = nmetrics->maxts.nts;
if (cont_metrics->hits > 0)
cont_metrics->avgts.nts = cont_metrics->cumts.nts / cont_metrics->hits;
free_gmetrics (nmetrics);
free (root);
return;
}
} else {
country_metrics = country_sub->metrics;
/* Handle mixed 2-level (persisted) and 3-level (live) data:
* older entries won't have a city sub-list. */
if (country_sub->sub_list == NULL)
country_sub->sub_list = new_gsublist ();
}
/* Accumulate metrics upward: city -> country -> continent */
country_metrics->hits += nmetrics->hits;
country_metrics->visitors += nmetrics->visitors;
country_metrics->nbw += nmetrics->nbw;
country_metrics->cumts.nts += nmetrics->cumts.nts;
if (nmetrics->maxts.nts > country_metrics->maxts.nts)
country_metrics->maxts.nts = nmetrics->maxts.nts;
if (country_metrics->hits > 0)
country_metrics->avgts.nts = country_metrics->cumts.nts / country_metrics->hits;
cont_metrics->hits += nmetrics->hits;
cont_metrics->visitors += nmetrics->visitors;
cont_metrics->nbw += nmetrics->nbw;
cont_metrics->cumts.nts += nmetrics->cumts.nts;
if (nmetrics->maxts.nts > cont_metrics->maxts.nts)
cont_metrics->maxts.nts = nmetrics->maxts.nts;
if (cont_metrics->hits > 0)
cont_metrics->avgts.nts = cont_metrics->cumts.nts / cont_metrics->hits;
/* Add city as sub-item under country (only if country's sub-list hasn't reached max_choices_sub) */
if (country_sub->sub_list->size < h->max_choices_sub) {
add_sub_item_back (country_sub->sub_list, h->module, nmetrics);
h->sub_items_size++;
} else {
free_gmetrics (nmetrics);
}
free (root);
}
#endif
/* Load raw data into our holder structure */
void
load_holder_data (GRawData *raw_data, GHolder *h, GModule module, GSort sort, uint32_t max_choices, uint32_t max_choices_sub) {
uint32_t i;
uint32_t size = 0;
uint32_t alloc_size = 0;
const GPanel *panel = panel_lookup (module);
/* Hierarchical panels group multiple raw items under fewer root items */
int is_hierarchical = (panel->insert == add_root_to_holder ||
#ifdef HAVE_GEOLOCATION
panel->insert == add_geo_to_holder ||
#endif
0);
#ifdef _DEBUG
clock_t begin = clock ();
double taken;
const char *modstr = NULL;
LOG_DEBUG (("== load_holder_data ==\n"));
#endif
size = raw_data->size;
/* For hierarchical data, we don't know how many root items we'll create,
* but it can't be more than size or max_choices */
alloc_size = size > max_choices ? max_choices : size;
h->holder_size = is_hierarchical ? size : (size > max_choices ? max_choices : size);
h->ht_size = size;
h->idx = 0;
h->module = module;
h->sub_items_size = 0;
h->max_choices = max_choices;
h->max_choices_sub = max_choices_sub;
h->items = new_gholder_item (alloc_size);
for (i = 0; i < h->holder_size; i++) {
panel->insert (raw_data->items[i], h, raw_data->type, panel);
}
sort_holder_items (h->items, h->idx, sort);
if (h->sub_items_size)
sort_sub_list (h, sort);
free_raw_data (raw_data);
#ifdef _DEBUG
modstr = get_module_str (module);
taken = (double) (clock () - begin) / CLOCKS_PER_SEC;
LOG_DEBUG (("== %-30s%f\n\n", modstr, taken));
#endif
}
src/gholder.h
+57
View File
@@ -0,0 +1,57 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef GHOLDER_H_INCLUDED
#define GHOLDER_H_INCLUDED
#define MTRC_ID_COUNTRY 0
#define MTRC_ID_CITY 1
#define MTRC_ID_ASN 2
#define MTRC_ID_HOSTNAME 3
#include "commons.h"
#include "sort.h"
/* Default Anonymization Levels */
typedef enum GAnonymizeLevels_ {
ANONYMIZE_DEFAULT = 1,
ANONYMIZE_STRONG,
ANONYMIZE_PEDANTIC,
} GAnonymizeLevels;
/* Function Prototypes */
GHolder *new_gholder (uint32_t size);
void *add_hostname_node (void *ptr_holder);
void free_holder_by_module (GHolder ** holder, GModule module);
void free_holder (GHolder ** holder);
void load_holder_data (GRawData * raw_data, GHolder * h, GModule module, GSort sort, uint32_t max_choices, uint32_t max_choices_sub);
void load_host_to_holder (GHolder * h, char *ip);
int dup_key_list (void *val, GSLList ** user_data);
#endif // for #ifndef GHOLDER_H
src/gkhash.c
+1653
View File
File diff suppressed because it is too large Load Diff
src/gkhash.h
+196
View File
@@ -0,0 +1,196 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef GKHASH_H_INCLUDED
#define GKHASH_H_INCLUDED
#include <stdint.h>
#include "gslist.h"
#include "gstorage.h"
#include "khash.h"
#include "parser.h"
#include "gkmhash.h"
#define DB_VERSION 2
#define DB_INSTANCE 1
typedef struct GKDB_ GKDB;
typedef struct GKHashStorage_ GKHashStorage;
/* *INDENT-OFF* */
/* uint32_t keys , GKDB payload */
KHASH_MAP_INIT_INT (igdb , GKDB *);
/* uint32_t keys , GKHashStorage payload */
KHASH_MAP_INIT_INT (igkh , GKHashStorage *);
/* uint32_t keys , uint32_t payload */
KHASH_MAP_INIT_INT (ii32 , uint32_t);
/* uint32_t keys , string payload */
KHASH_MAP_INIT_INT (is32 , char *);
/* uint32_t keys , uint64_t payload */
KHASH_MAP_INIT_INT (iu64 , uint64_t);
/* string keys , uint32_t payload */
KHASH_MAP_INIT_STR (si32 , uint32_t);
/* string keys , uint8_t payload */
KHASH_MAP_INIT_STR (si08 , uint8_t);
/* uint8_t keys , uint8_t payload */
KHASH_MAP_INIT_INT (ii08 , uint8_t);
/* string keys , string payload */
KHASH_MAP_INIT_STR (ss32 , char *);
/* uint64_t key , GLastParse payload */
KHASH_MAP_INIT_INT64 (iglp , GLastParse);
/* uint32_t keys , GSLList payload */
KHASH_MAP_INIT_INT (igsl , GSLList *);
/* string keys , uint64_t payload */
KHASH_MAP_INIT_STR (su64 , uint64_t);
/* uint64_t key , uint8_t payload */
KHASH_MAP_INIT_INT64 (u648 , uint8_t);
/* *INDENT-ON* */
/* Whole App Data store */
typedef struct GKHashDB_ {
GKHashMetric metrics[GAMTRC_TOTAL];
} GKHashDB;
/* DB */
struct GKDB_ {
GKHashDB *hdb; /* app-level hash tables */
Logs *logs; /* logs parsing per db instance */
GKHashModule *cache; /* cache modules */
GKHashStorage *store; /* per date OR module */
};
#define HT_FIRST_VAL(h, kvar, code) { khint_t __k; \
for (__k = kh_begin(h); __k != kh_end(h); ++__k) { \
if (!kh_exist(h,__k)) continue; \
(kvar) = kh_key(h,__k); \
code; \
} }
#define HT_SUM_VAL(h, kvar, code) { khint_t __k; \
for (__k = kh_begin(h); __k != kh_end(h); ++__k) { \
if (!kh_exist(h,__k)) continue; \
(kvar) = kh_key(h,__k); \
code; \
} }
#define HT_FOREACH_KEY(h, kvar, code) { khint_t __k; \
for (__k = kh_begin(h); __k != kh_end(h); ++__k) { \
if (!kh_exist(h,__k)) continue; \
(kvar) = kh_key(h,__k); \
code; \
} }
extern const GKHashMetric app_metrics[];
extern const size_t app_metrics_len;
/* *INDENT-OFF* */
void *new_igsl_ht (void);
void *new_ii08_ht (void);
void *new_ii32_ht (void);
void *new_is32_ht (void);
void *new_iu64_ht (void);
void *new_si32_ht (void);
void *new_su64_ht (void);
void *new_u648_ht (void);
void del_igsl_free (void *h, uint8_t free_data);
void del_ii08 (void *h, GO_UNUSED uint8_t free_data);
void del_ii32 (void *h, GO_UNUSED uint8_t free_data);
void del_is32_free (void *h, uint8_t free_data);
void del_iu64 (void *h, GO_UNUSED uint8_t free_data);
void del_si32_free (void *h, uint8_t free_data);
void del_su64_free (void *h, uint8_t free_data);
void del_u648 (void *h, GO_UNUSED uint8_t free_data);
void des_igsl_free (void *h, uint8_t free_data);
void des_ii08 (void *h, GO_UNUSED uint8_t free_data);
void des_ii32 (void *h, GO_UNUSED uint8_t free_data);
void des_is32_free (void *h, uint8_t free_data);
void des_iu64 (void *h, GO_UNUSED uint8_t free_data);
void des_si32_free (void *h, uint8_t free_data);
void des_su64_free (void *h, uint8_t free_data);
void des_u648 (void *h, GO_UNUSED uint8_t free_data);
int inc_iu64 (khash_t (iu64) * hash, uint32_t key, uint64_t inc);
int inc_su64 (khash_t (su64) * hash, const char *key, uint64_t inc);
int ins_iglp (khash_t (iglp) * hash, uint64_t key, const GLastParse *lp);
int ins_igsl (khash_t (igsl) * hash, uint32_t key, uint32_t value);
int ins_ii08 (khash_t (ii08) * hash, uint32_t key, uint8_t value);
int ins_ii32 (khash_t (ii32) * hash, uint32_t key, uint32_t value);
int ins_is32 (khash_t (is32) * hash, uint32_t key, char *value);
int ins_iu64 (khash_t (iu64) * hash, uint32_t key, uint64_t value);
int ins_si08 (khash_t (si08) * hash, const char *key, uint8_t value);
int ins_si32 (khash_t (si32) * hash, const char *key, uint32_t value);
int ins_su64 (khash_t (su64) * hash, const char *key, uint64_t value);
int ins_u648 (khash_t (u648) * hash, uint64_t key, uint8_t value);
uint32_t inc_ii32 (khash_t (ii32) * hash, uint32_t key, uint32_t inc);
uint32_t ins_ii32_ai (khash_t (ii32) * hash, uint32_t key);
uint32_t ins_ii32_inc (khash_t (ii32) * hash, uint32_t key, uint32_t (*cb) (khash_t (si32) *, const char *), khash_t (si32) * seqs, const char *seqk);
uint32_t ins_si32_inc (khash_t (si32) * hash, const char *key, uint32_t (*cb) (khash_t (si32) *, const char *), khash_t (si32) * seqs, const char *seqk);
char *get_is32 (khash_t (is32) * hash, uint32_t key);
uint32_t get_ii32 (khash_t (ii32) * hash, uint32_t key);
uint32_t get_si32 (khash_t (si32) * hash, const char *key);
uint64_t get_iu64 (khash_t (iu64) * hash, uint32_t key);
uint64_t get_su64 (khash_t (su64) * hash, const char *key);
uint8_t get_ii08 (khash_t (ii08) * hash, uint32_t key);
uint8_t get_si08 (khash_t (si08) * hash, const char *key);
void get_ii32_min_max (khash_t (ii32) * hash, uint32_t * min, uint32_t * max);
void get_iu64_min_max (khash_t (iu64) * hash, uint64_t * min, uint64_t * max);
int ht_insert_hostname (const char *ip, const char *host);
int ht_insert_json_logfmt (GO_UNUSED void *userdata, char *key, char *spec);
int ht_insert_last_parse (uint64_t key, const GLastParse *lp);
uint32_t ht_inc_cnt_overall (const char *key, uint32_t val);
uint32_t ht_ins_seq (khash_t (si32) * hash, const char *key);
uint8_t ht_insert_meth_proto (const char *key);
char *ht_get_hostname (const char *host);
char *ht_get_json_logfmt (const char *key);
uint32_t ht_get_excluded_ips (void);
uint32_t ht_get_invalid (void);
uint32_t ht_get_processed (void);
uint32_t ht_get_processing_time (void);
uint8_t get_method_proto (const char *value);
uint32_t *get_sorted_dates (uint32_t * len);
void free_storage (void);
void init_pre_storage (Logs *logs);
const char *get_mtr_type_str (GSMetricType type);
void *get_db_instance (uint32_t key);
void *get_hdb (GKDB * db, GAMetric mtrc);
GLastParse ht_get_last_parse (uint64_t key);
Logs *get_db_logs(uint32_t instance);
/* *INDENT-ON* */
#endif // for #ifndef GKHASH_H
src/gkmhash.c
+1465
View File
File diff suppressed because it is too large Load Diff
src/gkmhash.h
+275
View File
@@ -0,0 +1,275 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef GKMHASH_H_INCLUDED
#define GKMHASH_H_INCLUDED
#include "gstorage.h"
typedef struct GKHashMetric_ GKHashMetric;
/* Data store per module */
typedef struct GKHashModule_ {
GModule module;
GKHashMetric metrics[GSMTRC_TOTAL];
} GKHashModule;
/* Data store global */
typedef struct GKHashGlobal_ {
GKHashMetric metrics[GSMTRC_TOTAL];
} GKHashGlobal;
struct GKHashStorage_ {
GKHashModule *mhash; /* modules */
GKHashGlobal *ghash; /* global */
};
/* Metrics Storage */
/* Most metrics are encapsulated within a GKHashStorage structure, which is
* conformed of a dated key and a GKHashStorage struct value. This helps to
* easily destroy the entire dated storage at any time. */
/* GLOBAL METRICS */
/* ============== */
/* Maps a string key containing an IP|DATE|UA(hash uint32_t => hex) to an
* autoincremented value.
*
* 192.168.0.1|27/Apr/2020|7E8E0E -> 1
* 192.168.0.1|28/Apr/2020|7E8E0E -> 2
*/
/*khash_t(si32) MTRC_UNIQUE_KEYS */
/* Maps string keys made out of the user agent to an autoincremented value.
*
* Debian APT-HTTP/1.3 (1.0.9.8.5) -> 1838302 -> 1
* Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1;) -> 8723842 -> 2
*/
/*khash_t(ii32) MTRC_AGENT_KEYS */
/* Maps integer keys from the autoincremented MTRC_AGENT_KEYS value to the user
* agent.
*
* 1 -> Debian APT-HTTP/1.3 (1.0.9.8.5)
* 2 -> Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)
*/
/*khash_t(is32) MTRC_AGENT_VALS */
/* Maps a single numeric key (usually 1) to an autoincremented hits value.
*
* 1 -> 5
*/
/*khash_t(is32) MTRC_CNT_VALID */
/* Maps a single numeric key (usually 1) to an autoincremented bw value.
*
* 1 -> 592933
*/
/*khash_t(iu64) MTRC_CNT_BW */
/* MODULE METRICS */
/* ============== */
/* Maps keys (string) to a hash to a numeric values (uint32_t).
* this mitigates the issue of having multiple stores
* with the same string key, and therefore, avoids unnecessary
* memory usage (in most cases).
*
* HEAD|/index.php -> 9872347 -> 1
* POST|/index.php -> 3452345 -> 2
* Windows XP -> 2842343 -> 3
* Ubuntu 10.10 -> 1852342 -> 4
* GET|Ubuntu 10.10-> 4872343 -> 5
* GNU+Linux -> 5862347 -> 6
* 26/Dec/2014 -> 9874347 -> 7
* Windows -> 3875347 -> 8
*/
/*khash_t(si32) MTRC_KEYMAP */
/* Maps integer keys of root elements from the keymap hash
* to actual string values.
*
* 6 -> GNU+Linux
* 8 -> Windows
*/
/*khash_t(is32) MTRC_ROOTMAP */
/* Maps integer keys of data elements from the keymap hash
* to actual string values.
*
* 1 -> /index.php
* 2 -> /index.php
* 3 -> Windows xp
* 4 -> Ubuntu 10.10
* 5 -> Ubuntu 10.10
* 7 -> 26/dec/2014
*/
/*khash_t(is32) MTRC_DATAMAP */
/* Maps the unique uint32_t key of the IP/date/UA and the uint32_t key from the
* data field encoded into a uint64_t key to numeric autoincremented values.
* e.g., "1&4" => 12938238293 to ai.
*
* 201023232 -> 1
* 202939232 -> 2
*/
/*khash_t(si32) MTRC_UNIQMAP */
/* Maps integer keys made from a data key to an integer root key in
* MTRC_KEYMAP.
*
* 4 -> 6
* 3 -> 8
*/
/*khash_t(ii32) MTRC_ROOT */
/* Maps integer key from the keymap hash to the number of
* hits.
*
* 1 -> 10934
* 2 -> 3231
* 3 -> 500
* 4 -> 201
* 5 -> 206
*/
/*khash_t(ii32) MTRC_HITS */
/* Maps numeric keys made from the uniqmap store to autoincremented values
* (counter).
* 10 -> 100
* 40 -> 56
*/
/*khash_t(ii32) MTRC_VISITORS */
/* Maps numeric data keys to bandwidth (in bytes).
* 1 -> 1024
* 2 -> 2048
*/
/*khash_t(iu64) MTRC_BW */
/* Maps numeric data keys to cumulative time served (in usecs/msecs).
* 1 -> 187
* 2 -> 208
*/
/*khash_t(iu64) MTRC_CUMTS */
/* Maps numeric data keys to max time served (in usecs/msecs).
* 1 -> 1287
* 2 -> 2308
*/
/*khash_t(iu64) MTRC_MAXTS */
/* Maps numeric data keys to uint8_t values.
* 1 -> 3
* 2 -> 4
*/
/*khash_t(is32) MTRC_METHODS */
/* Maps numeric data keys to uint8_t values.
* 1 -> 1
* 2 -> 1
*/
/*khash_t(is32) MTRC_PROTOCOLS */
/* Maps numeric unique data keys (e.g., 192.168.0.1 => 1) to the unique user
* agent key. Therefore, 1 IP can contain multiple user agents
* 1 -> 3,5
* 2 -> 4,5,6,8
*/
/*khash_t(igsl) MTRC_AGENTS */
/* Maps a string key counter such as sum of hits to an autoincremented value
* "sum_hits" -> 9383
* "sum_bw" -> 3232932
*/
/*khash_t(igsl) MTRC_METADATA */
/* *INDENT-OFF* */
extern const GKHashMetric module_metrics[];
extern const GKHashMetric global_metrics[];
extern const size_t global_metrics_len;
extern const size_t module_metrics_len;
char *ht_get_datamap (GModule module, uint32_t key);
char *ht_get_host_agent_val (uint32_t key);
char *ht_get_method (GModule module, uint32_t key);
char *ht_get_protocol (GModule module, uint32_t key);
char *ht_get_root (GModule module, uint32_t key);
uint32_t ht_get_hits (GModule module, int key);
uint32_t ht_get_keymap (GModule module, const char *key);
uint32_t ht_get_size_datamap (GModule module);
uint32_t ht_get_size_dates (void);
uint32_t ht_get_size_uniqmap (GModule module);
uint32_t ht_get_visitors (GModule module, uint32_t key);
uint32_t ht_sum_valid (void);
uint64_t ht_get_bw (GModule module, uint32_t key);
uint64_t ht_get_cumts (GModule module, uint32_t key);
uint64_t ht_get_maxts (GModule module, uint32_t key);
uint64_t ht_get_meta_data (GModule module, const char *key);
uint64_t ht_sum_bw (void);
void *get_hash (int module, uint64_t key, GSMetric metric);
void ht_get_bw_min_max (GModule module, uint64_t * min, uint64_t * max);
void ht_get_cumts_min_max (GModule module, uint64_t * min, uint64_t * max);
void ht_get_hits_min_max (GModule module, uint32_t * min, uint32_t * max);
void ht_get_maxts_min_max (GModule module, uint64_t * min, uint64_t * max);
void ht_get_visitors_min_max (GModule module, uint32_t * min, uint32_t * max);
int ht_inc_cnt_bw (uint32_t date, uint64_t inc);
int ht_insert_agent (GModule module, uint32_t date, uint32_t key, uint32_t value);
int ht_insert_agent_value (uint32_t date, uint32_t key, char *value);
int ht_insert_bw (GModule module, uint32_t date, uint32_t key, uint64_t inc, uint32_t ckey);
int ht_insert_cumts (GModule module, uint32_t date, uint32_t key, uint64_t inc, uint32_t ckey);
int ht_insert_datamap (GModule module, uint32_t date, uint32_t key, const char *value, uint32_t ckey);
int ht_insert_date (uint32_t key);
int ht_insert_maxts (GModule module, uint32_t date, uint32_t key, uint64_t value, uint32_t ckey);
int ht_insert_method (GModule module, uint32_t date, uint32_t key, const char *value, uint32_t ckey);
int ht_insert_protocol (GModule module, uint32_t date, uint32_t key, const char *value, uint32_t ckey);
int ht_insert_root (GModule module, uint32_t date, uint32_t key, uint32_t value, uint32_t dkey, uint32_t rkey);
int ht_insert_rootmap (GModule module, uint32_t date, uint32_t key, const char *value, uint32_t ckey);
int ht_insert_uniqmap (GModule module, uint32_t date, uint32_t key, uint32_t value);
uint32_t ht_inc_cnt_valid (uint32_t date, uint32_t inc);
uint32_t ht_insert_agent_key (uint32_t date, uint32_t key);
uint32_t ht_insert_hits (GModule module, uint32_t date, uint32_t key, uint32_t inc, uint32_t ckey);
uint32_t ht_insert_keymap (GModule module, uint32_t date, uint32_t key, uint32_t * ckey);
uint32_t ht_insert_unique_key (uint32_t date, const char *key);
uint32_t ht_insert_visitor (GModule module, uint32_t date, uint32_t key, uint32_t inc, uint32_t ckey);
int ht_insert_meta_data (GModule module, uint32_t date, const char *key, uint64_t value);
int invalidate_date (int date);
int rebuild_rawdata_cache (void);
void des_igkh (void *h);
void free_cache (GKHashModule * cache);
void init_storage (void);
GRawData *parse_raw_data (GModule module);
GSLList *ht_get_host_agent_list (GModule module, uint32_t key);
GSLList *ht_get_keymap_list_from_key (GModule module, uint32_t key);
/* *INDENT-ON* */
#endif // for #ifndef GKMHASH_H
src/gmenu.c
+145
View File
@@ -0,0 +1,145 @@
/**
* gmenu.c -- goaccess menus
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "gmenu.h"
#include "xmalloc.h"
#include "ui.h"
/* Allocate memory for a new GMenu instance.
*
* On success, the newly allocated GMenu is returned . */
GMenu *
new_gmenu (WINDOW *parent, int h, int w, int y, int x) {
GMenu *menu = xmalloc (sizeof (GMenu));
memset (menu, 0, sizeof *menu);
menu->count = 0;
menu->idx = 0;
menu->multiple = 0;
menu->selectable = 0;
menu->start = 0;
menu->status = 0;
menu->h = h;
menu->w = w;
menu->x = x;
menu->y = y;
menu->win = derwin (parent, menu->h, menu->w, menu->y, menu->x);
return menu;
}
/* Render actual menu item */
static void
draw_menu_item (GMenu *menu, char *s, int x, int y, int w, int checked, GColors *(*func) (void)) {
char check, *lbl = NULL;
if (menu->selectable) {
check = checked ? 'x' : ' ';
lbl = xmalloc (snprintf (NULL, 0, "[%c] %s", check, s) + 1);
sprintf (lbl, "[%c] %s", check, s);
draw_header (menu->win, lbl, "%s", y, x, w, (*func));
free (lbl);
} else {
draw_header (menu->win, s, "%s", y, x, w, (*func));
}
}
/* Displays a menu to its associated window.
*
* On error, 1 is returned.
* On success, the newly created menu is added to the window and 0 is
* returned. */
int
post_gmenu (GMenu *menu) {
GColors *(*func) (void);
int i = 0, j = 0, start, end, height, total, checked = 0;
if (menu == NULL)
return 1;
werase (menu->win);
height = menu->h;
start = menu->start;
total = menu->size;
end = height < total ? start + height : total;
for (i = start; i < end; i++, j++) {
func = i == menu->idx ? color_selected : color_default;
checked = menu->items[i].checked ? 1 : 0;
draw_menu_item (menu, menu->items[i].name, 0, j, menu->w, checked, func);
}
wrefresh (menu->win);
return 0;
}
/* Main work horse of the menu system processing input events */
void
gmenu_driver (GMenu *menu, int c) {
int i;
switch (c) {
case REQ_DOWN:
if (menu->idx >= menu->size - 1)
break;
++menu->idx;
if (menu->idx >= menu->h && menu->idx >= menu->start + menu->h)
menu->start++;
post_gmenu (menu);
break;
case REQ_UP:
if (menu->idx <= 0)
break;
--menu->idx;
if (menu->idx < menu->start)
--menu->start;
post_gmenu (menu);
break;
case REQ_SEL:
if (!menu->multiple) {
for (i = 0; i < menu->size; i++)
menu->items[i].checked = 0;
}
if (menu->items[menu->idx].checked)
menu->items[menu->idx].checked = 0;
else
menu->items[menu->idx].checked = 1;
post_gmenu (menu);
break;
}
}
src/gmenu.h
+84
View File
@@ -0,0 +1,84 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#ifdef HAVE_NCURSESW_NCURSES_H
#include <ncursesw/ncurses.h>
#elif HAVE_NCURSES_NCURSES_H
#include <ncurses/ncurses.h>
#elif HAVE_NCURSES_H
#include <ncurses.h>
#elif HAVE_CURSES_H
#include <curses.h>
#endif
#ifndef GMENU_H_INCLUDED
#define GMENU_H_INCLUDED
enum ACTION {
REQ_DOWN,
REQ_UP,
REQ_SEL
};
typedef struct GMenu_ GMenu;
typedef struct GItem_ GItem;
/* Menu Item */
struct GItem_ {
char *name;
int checked;
};
/* Menu Panel */
struct GMenu_ {
WINDOW *win;
int count;
int size;
int idx;
int start;
int h;
int w;
int x;
int y;
unsigned short multiple;
unsigned short selectable;
unsigned short status;
GItem *items;
};
GMenu *new_gmenu (WINDOW * parent, int h, int w, int y, int x);
int post_gmenu (GMenu * menu);
void gmenu_driver (GMenu * menu, int c);
#endif
src/goaccess.c
+1900
View File
File diff suppressed because it is too large Load Diff
src/goaccess.h
+43
View File
@@ -0,0 +1,43 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef GOACCESS_H_INCLUDED
#define GOACCESS_H_INCLUDED
#include "ui.h"
#include "dialogs.h"
#define RAND_FN 7 + 1
extern GSpinner *parsing_spinner;
extern int active_gdns; /* kill dns pthread flag */
void read_client (void *ptr_data);
#endif
src/gslist.c
+173
View File
@@ -0,0 +1,173 @@
/**
* gslist.c -- A Singly link list implementation
* _______ _______ __ __
* / ____/ | / / ___/____ _____/ /_____ / /_
* / / __ | | /| / /\__ \/ __ \/ ___/ //_/ _ \/ __/
* / /_/ / | |/ |/ /___/ / /_/ / /__/ ,< / __/ /_
* \____/ |__/|__//____/\____/\___/_/|_|\___/\__/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include "gslist.h"
#include "gstorage.h"
#include "xmalloc.h"
/* Instantiate a new Singly linked-list node.
*
* On error, aborts if node can't be malloc'd.
* On success, the GSLList node. */
GSLList *
list_create (void *data) {
GSLList *node = xmalloc (sizeof (GSLList));
node->data = data;
node->next = NULL;
return node;
}
/* Create and insert a node after a given node.
*
* On error, aborts if node can't be malloc'd.
* On success, the newly created node. */
GSLList *
list_insert_append (GSLList *node, void *data) {
GSLList *newnode;
newnode = list_create (data);
newnode->next = node->next;
node->next = newnode;
return newnode;
}
/* Create and insert a node in front of the list.
*
* On error, aborts if node can't be malloc'd.
* On success, the newly created node. */
GSLList *
list_insert_prepend (GSLList *list, void *data) {
GSLList *newnode;
newnode = list_create (data);
newnode->next = list;
return newnode;
}
/* Find a node given a pointer to a function that compares them.
*
* If comparison fails, NULL is returned.
* On success, the existing node is returned. */
GSLList *
list_find (GSLList *node, int (*func) (void *, void *), void *data) {
while (node) {
if (func (node->data, data) > 0)
return node;
node = node->next;
}
return NULL;
}
GSLList *
list_copy (GSLList *node) {
GSLList *list = NULL;
while (node) {
if (!list)
list = list_create (i322ptr ((*(uint32_t *) node->data)));
else
list = list_insert_prepend (list, i322ptr ((*(uint32_t *) node->data)));
node = node->next;
}
return list;
}
/* Remove all nodes from the list.
*
* On success, 0 is returned. */
int
list_remove_nodes (GSLList *list) {
GSLList *tmp;
while (list != NULL) {
tmp = list->next;
if (list->data)
free (list->data);
free (list);
list = tmp;
}
return 0;
}
/* Remove the given node from the list.
*
* On error, 1 is returned.
* On success, 0 is returned. */
int
list_remove_node (GSLList **list, GSLList *node) {
GSLList **current = list, *next = NULL;
for (; *current; current = &(*current)->next) {
if ((*current) != node)
continue;
next = (*current)->next;
if ((*current)->data)
free ((*current)->data);
free (*current);
*current = next;
return 0;
}
return 1;
}
/* Iterate over the single linked-list and call function pointer.
*
* If function pointer does not return 0, -1 is returned.
* On success, 0 is returned. */
int
list_foreach (GSLList *node, int (*func) (void *, void *), void *user_data) {
while (node) {
if (func (node->data, user_data) != 0)
return -1;
node = node->next;
}
return 0;
}
/* Count the number of elements on the linked-list.
*
* On success, the number of elements is returned. */
int
list_count (GSLList *node) {
int count = 0;
while (node != 0) {
count++;
node = node->next;
}
return count;
}
src/gslist.h
+58
View File
@@ -0,0 +1,58 @@
/**
* _______ _______ __ __
* / ____/ | / / ___/____ _____/ /_____ / /_
* / / __ | | /| / /\__ \/ __ \/ ___/ //_/ _ \/ __/
* / /_/ / | |/ |/ /___/ / /_/ / /__/ ,< / __/ /_
* \____/ |__/|__//____/\____/\___/_/|_|\___/\__/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef GSLIST_H_INCLUDED
#define GSLIST_H_INCLUDED
/* Generic Single linked-list */
typedef struct GSLList_ {
void *data;
struct GSLList_ *next;
} GSLList;
#define GSLIST_FOREACH(node, data, code) { \
GSLList *__tmp = node; \
while (__tmp) { \
(data) = __tmp->data; \
code; \
__tmp = __tmp->next; \
}}
/* single linked-list */
GSLList *list_create (void *data);
GSLList *list_find (GSLList * node, int (*func) (void *, void *), void *data);
GSLList *list_insert_append (GSLList * node, void *data);
GSLList *list_insert_prepend (GSLList * list, void *data);
GSLList *list_copy (GSLList * node);
int list_count (GSLList * list);
int list_foreach (GSLList * node, int (*func) (void *, void *), void *user_data);
int list_remove_node (GSLList ** list, GSLList * node);
int list_remove_nodes (GSLList * list);
#endif // for #ifndef GSLIST_H
src/gstorage.c
+1642
View File
File diff suppressed because it is too large Load Diff
src/gstorage.h
+167
View File
@@ -0,0 +1,167 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef GSTORAGE_H_INCLUDED
#define GSTORAGE_H_INCLUDED
#include "commons.h"
#include "parser.h"
#define DB_PATH "/tmp"
#define GAMTRC_TOTAL 8
/* Enumerated App Metrics */
typedef enum GAMetric_ {
MTRC_DATES,
MTRC_SEQS,
MTRC_CNT_OVERALL,
MTRC_HOSTNAMES,
MTRC_LAST_PARSE,
MTRC_JSON_LOGFMT,
MTRC_METH_PROTO,
MTRC_DB_PROPS,
} GAMetric;
/* Enumerated Storage Metrics */
typedef enum GSMetricType_ {
/* uint32_t key - uint32_t val */
MTRC_TYPE_II32,
/* uint32_t key - string val */
MTRC_TYPE_IS32,
/* uint32_t key - uint64_t val */
MTRC_TYPE_IU64,
/* string key - uint32_t val */
MTRC_TYPE_SI32,
/* string key - uint8_t val */
MTRC_TYPE_SI08,
/* uint32_t key - uint8_t val */
MTRC_TYPE_II08,
/* string key - string val */
MTRC_TYPE_SS32,
/* uint32_t key - GSLList val */
MTRC_TYPE_IGSL,
/* string key - uint64_t val */
MTRC_TYPE_SU64,
/* uint32_t key - GKHashStorage_ val */
MTRC_TYPE_IGKH,
/* uint64_t key - uint32_t val */
MTRC_TYPE_U648,
/* uint64_t key - GLastParse val */
MTRC_TYPE_IGLP,
} GSMetricType;
typedef struct GKHashMetric_ {
union {
GSMetric storem;
GAMetric dbm;
} metric;
GSMetricType type;
void *(*alloc) (void);
void (*des) (void *, uint8_t free_data);
void (*del) (void *, uint8_t free_data);
uint8_t free_data:1;
void *hash;
const char *filename;
} GKHashMetric;
/* Each record contains a data value, i.e., Windows XP, and it may contain a
* root value, i.e., Windows, and a unique key which is the combination of
* date, IP and user agent */
typedef struct GKeyData_ {
const void *data;
uint32_t dhash;
uint32_t data_nkey;
uint32_t cdnkey; /* cache data nkey */
uint32_t rhash;
const void *root;
const void *root_key;
uint32_t root_nkey;
uint32_t crnkey; /* cache root nkey */
void *uniq_key;
uint32_t uniq_nkey;
uint32_t numdate;
} GKeyData;
typedef struct GParse_ {
GModule module;
int (*key_data) (GKeyData * kdata, GLogItem * logitem);
/* data field */
void (*datamap) (GModule module, GKeyData * kdata);
void (*rootmap) (GModule module, GKeyData * kdata);
void (*hits) (GModule module, GKeyData * kdata);
void (*visitor) (GModule module, GKeyData * kdata);
void (*bw) (GModule module, GKeyData * kdata, uint64_t size);
void (*cumts) (GModule module, GKeyData * kdata, uint64_t ts);
void (*maxts) (GModule module, GKeyData * kdata, uint64_t ts);
void (*method) (GModule module, GKeyData * kdata, const char *data);
void (*protocol) (GModule module, GKeyData * kdata, const char *data);
void (*agent) (GModule module, GKeyData * kdata, uint32_t agent_nkey);
} GParse;
typedef struct httpmethods_ {
const char *method;
int len;
} httpmethods;
typedef struct httpprotocols_ {
const char *protocol;
int len;
} httpprotocols;
extern const httpmethods http_methods[];
extern const httpprotocols http_protocols[];
extern const size_t http_methods_len;
extern const size_t http_protocols_len;
const char *get_mtr_str (GSMetric metric);
int excluded_ip (GLogItem * logitem);
uint32_t *i322ptr (uint32_t val);
uint64_t *uint642ptr (uint64_t val);
void count_process_and_invalid (GLog * glog, GLogItem * logitem, const char *line);
void count_process (GLog * glog);
void free_gmetrics (GMetrics * metric);
void insert_methods_protocols (void);
void process_log (GLogItem * logitem);
void set_browser_os (GLogItem * logitem);
void set_data_metrics (GMetrics * ometrics, GMetrics ** nmetrics, GPercTotals totals);
void set_module_totals (GPercTotals * totals);
void uncount_invalid (GLog * glog);
void uncount_processed (GLog * glog);
GMetrics *new_gmetrics (void);
#ifdef HAVE_GEOLOCATION
const char *get_continent_for_country (const char *country);
void free_country_continent_map (void);
#endif
#endif // for #ifndef GSTORAGE_H
src/gwsocket.c
+545
View File
@@ -0,0 +1,545 @@
/**
* gwsocket.c -- An interface to send/recv data from/to Web Socket Server
* _______ _______ __ __
* / ____/ | / / ___/____ _____/ /_____ / /_
* / / __ | | /| / /\__ \/ __ \/ ___/ //_/ _ \/ __/
* / /_/ / | |/ |/ /___/ / /_/ / /__/ ,< / __/ /_
* \____/ |__/|__//____/\____/\___/_/|_|\___/\__/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <fcntl.h>
#include <unistd.h>
#include "gwsocket.h"
#include "commons.h"
#include "error.h"
#include "goaccess.h"
#include "pdjson.h"
#include "json.h"
#include "settings.h"
#include "websocket.h"
#include "wsauth.h"
#include "xmalloc.h"
/* Allocate memory for a new GWSReader instance.
*
* On success, the newly allocated GWSReader is returned. */
GWSReader *
new_gwsreader (void) {
GWSReader *reader = xmalloc (sizeof (GWSReader));
memset (reader, 0, sizeof *reader);
return reader;
}
/* Allocate memory for a new GWSWriter instance.
*
* On success, the newly allocated GWSWriter is returned. */
GWSWriter *
new_gwswriter (void) {
GWSWriter *writer = xmalloc (sizeof (GWSWriter));
memset (writer, 0, sizeof *writer);
return writer;
}
/* Write the JSON data to a pipe.
*
* If unable to write bytes, -1 is returned.
* On success, the number of written bytes is returned . */
static int
write_holder (int fd, const char *buf, int len) {
int i, ret = 0;
for (i = 0; i < len;) {
ret = write (fd, buf + i, len - i);
if (ret < 0) {
if (errno == EINTR || errno == EAGAIN)
continue;
return -1;
} else {
i += ret;
}
}
return i;
}
/* Clear an incoming FIFO packet and header data. */
static void
clear_fifo_packet (GWSReader *gwserver) {
memset (gwserver->hdr, 0, sizeof (gwserver->hdr));
gwserver->hlen = 0;
if (gwserver->packet == NULL)
return;
if (gwserver->packet->data)
free (gwserver->packet->data);
free (gwserver->packet);
gwserver->packet = NULL;
}
/* Pack the JSON data into a network byte order and writes it to a
* pipe.
*
* On success, 0 is returned . */
int
broadcast_holder (int fd, const char *buf, int len) {
char *p = NULL, *ptr = NULL;
p = calloc (sizeof (uint32_t) * 3, sizeof (char));
ptr = p;
ptr += pack_uint32 (ptr, 0);
ptr += pack_uint32 (ptr, 0x01);
ptr += pack_uint32 (ptr, len);
write_holder (fd, p, sizeof (uint32_t) * 3);
write_holder (fd, buf, len);
free (p);
return 0;
}
/* Pack the JSON data into a network byte order and write it to a
* pipe.
*
* On success, 0 is returned . */
int
send_holder_to_client (int fd, int listener, const char *buf, int len) {
char *p = NULL, *ptr = NULL;
p = calloc (sizeof (uint32_t) * 3, sizeof (char));
ptr = p;
ptr += pack_uint32 (ptr, listener);
ptr += pack_uint32 (ptr, 0x01);
ptr += pack_uint32 (ptr, len);
write_holder (fd, p, sizeof (uint32_t) * 3);
write_holder (fd, buf, len);
free (p);
return 0;
}
/* Attempt to read data from the named pipe on strict mode.
* Note: For now it only reads on new connections, i.e., onopen.
*
* If there's less data than requested, 0 is returned
* If the thread is done, 1 is returned */
int
read_fifo (GWSReader *gwsreader, void (*f) (int)) {
WSPacket **pa = &gwsreader->packet;
char *ptr;
int bytes = 0, readh = 0, need = 0, fd = gwsreader->fd;
uint32_t listener = 0, type = 0, size = 0;
struct pollfd fds[] = {
{.fd = gwsreader->self_pipe[0],.events = POLLIN},
{.fd = gwsreader->fd,.events = POLLIN,},
};
if (poll (fds, sizeof (fds) / sizeof (fds[0]), -1) == -1) {
switch (errno) {
case EINTR:
break;
default:
FATAL ("Unable to poll: %s.", strerror (errno));
}
}
/* handle self-pipe trick */
if (fds[0].revents & POLLIN)
return 1;
if (!(fds[1].revents & POLLIN)) {
LOG (("No file descriptor set on read_message()\n"));
return 0;
}
readh = gwsreader->hlen; /* read from header so far */
need = HDR_SIZE - readh; /* need to read */
if (need > 0) {
if ((bytes = ws_read_fifo (fd, gwsreader->hdr, &gwsreader->hlen, readh, need)) < 0)
return 0;
if (bytes != need)
return 0;
}
/* unpack size, and type */
ptr = gwsreader->hdr;
ptr += unpack_uint32 (ptr, &listener);
ptr += unpack_uint32 (ptr, &type);
ptr += unpack_uint32 (ptr, &size);
if ((*pa) == NULL) {
(*pa) = xcalloc (1, sizeof (WSPacket));
(*pa)->type = type;
(*pa)->size = size;
(*pa)->data = xcalloc (size + 1, sizeof (char));
}
readh = (*pa)->len; /* read from payload so far */
need = (*pa)->size - readh; /* need to read */
if (need > 0) {
if ((bytes = ws_read_fifo (fd, (*pa)->data, &(*pa)->len, readh, need)) < 0)
return 0;
if (bytes != need)
return 0;
}
clear_fifo_packet (gwsreader);
/* fast forward JSON data to the given client */
(*f) (listener);
return 0;
}
/* Callback once a new connection is established
*
* It writes to a named pipe a header containing the socket, the
* message type, the payload's length and the actual payload */
static int
onopen (WSPipeOut *pipeout, WSClient *client) {
uint32_t hsize = sizeof (uint32_t) * 3;
char *hdr = calloc (hsize, sizeof (char));
char *ptr = hdr;
ptr += pack_uint32 (ptr, client->listener);
ptr += pack_uint32 (ptr, WS_OPCODE_TEXT);
ptr += pack_uint32 (ptr, INET6_ADDRSTRLEN);
ws_write_fifo (pipeout, hdr, hsize);
ws_write_fifo (pipeout, client->remote_ip, INET6_ADDRSTRLEN);
free (hdr);
return 0;
}
/* validate_token_message()
*
* Parses a JSON payload (assumed to be null-terminated) that should have
* the form: {"action":"validate_token","token":"..."}
* If the action is "validate_token", the token is verified using verify_jwt_token().
* If verification is successful, the client's stored JWT is updated.
*
* Returns:
* 1 if the message was a token validation message and authentication succeeded.
* 0 if the message is not a token validation message.
* -1 if an error occurs (including validation failure).
*/
#ifdef HAVE_LIBSSL
static int
validate_token_message (const char *payload, WSClient *client) {
json_stream json;
enum json_type t = JSON_ERROR;
size_t len = 0, level = 0;
enum json_type ctx = JSON_ERROR;
char *curr_key = NULL;
char *action = NULL;
char *token = NULL;
json_open_string (&json, payload);
json_set_streaming (&json, false);
/* Expect a JSON object */
t = json_next (&json);
if (t != JSON_OBJECT) {
json_close (&json);
return -1;
}
/* Iterate over the JSON tokens */
while ((t = json_next (&json)) != JSON_DONE && t != JSON_ERROR) {
ctx = json_get_context (&json, &level);
/* When (level % 2) != 0 and not in an array, the token is a key */
if ((level % 2) != 0 && ctx != JSON_ARRAY) {
if (curr_key)
free (curr_key);
curr_key = xstrdup (json_get_string (&json, &len));
} else {
/* Otherwise, token is a value for the last encountered key */
if (curr_key) {
char *val = xstrdup (json_get_string (&json, &len));
if (strcmp (curr_key, "action") == 0) {
action = val;
} else if (strcmp (curr_key, "token") == 0) {
token = val;
} else {
free (val);
}
free (curr_key);
curr_key = NULL;
}
}
}
if (curr_key)
free (curr_key);
json_close (&json);
/* If action is not "validate_token", then this message is not for token validation */
if (!action || strcmp (action, "validate_token") != 0) {
if (action)
free (action);
if (token)
free (token);
return 0;
}
/* For token validation, the token must exist */
if (!token) {
LOG (("Missing token in validate_token message from client %d [%s]\n", client->listener,
client->remote_ip));
free (action);
return -1;
}
/* Verify the token using the configured secret */
if (conf.ws_auth_secret && verify_jwt_token (token, conf.ws_auth_secret) != 1) {
LOG (("Authentication failed for client %d [%s]\n", client->listener, client->remote_ip));
free (action);
free (token);
client->status = WS_ERR | WS_CLOSE;
return -1;
}
/* Authentication succeeded: update client's stored token */
if (client->headers->jwt)
free (client->headers->jwt);
client->headers->jwt = strdup (token);
LOG (("Token validated and updated for client %d [%s]\n", client->listener, client->remote_ip));
free (action);
free (token);
return 1;
}
#endif
/* onmessage()
*
* Entry point for incoming messages. This function first checks if the message
* is a text message and ensures that the payload is null-terminated (so JSON
* parsing will work correctly). It then delegates token validation to
* validate_token_message(). In the future, onmessage() may handle other message
* types.
*/
#ifdef HAVE_LIBSSL
static int
onmessage (GO_UNUSED WSPipeOut *pipeout, WSClient *client) {
int ret = 1;
char *payload = client->message->payload;
int allocated = 0;
/* If this is a text message, ensure the payload is null-terminated.
* Binary frames should not be modified.
*/
if (client->message->opcode == WS_OPCODE_TEXT) {
if (memchr (payload, '\0', client->message->payloadsz) == NULL) {
char *tmp = malloc (client->message->payloadsz + 1);
if (!tmp) {
LOG (("Memory allocation error for client %d [%s]\n", client->listener, client->remote_ip));
return -1;
}
memcpy (tmp, payload, client->message->payloadsz);
tmp[client->message->payloadsz] = '\0';
payload = tmp;
allocated = 1;
}
}
/* Delegate processing to validate_token_message().
* In the future, you can add additional branches for other message types.
*/
ret = validate_token_message (payload, client);
if (allocated)
free (payload);
return ret;
}
#endif
/* Done parsing, clear out line and set status message. */
void
set_ready_state (void) {
fprintf (stderr, "\33[2K\r");
fprintf (stderr, "%s\n", INFO_WS_READY_FOR_CONN);
}
/* Open the named pipe where the websocket server writes to.
*
* If unable to open, -1 is returned.
* On success, return the new file descriptor is returned . */
int
open_fifoout (void) {
const char *fifo = conf.fifo_out;
int fdfifo;
/* open fifo for reading before writing */
ws_setfifo (fifo);
if ((fdfifo = open (fifo, O_RDWR | O_NONBLOCK)) == -1)
return -1;
return fdfifo;
}
/* Open the named pipe where the websocket server reads from.
*
* If unable to open, -1 is returned.
* On success, return the new file descriptor is returned . */
int
open_fifoin (void) {
const char *fifo = conf.fifo_in;
int fdfifo;
if ((fdfifo = open (fifo, O_WRONLY | O_NONBLOCK)) == -1)
return -1;
return fdfifo;
}
/* Set the self-pipe trick to handle poll(2). */
void
set_self_pipe (int *self_pipe) {
/* Initialize self pipe. */
if (pipe (self_pipe) == -1)
FATAL ("Unable to create pipe: %s.", strerror (errno));
/* make the read and write pipe non-blocking */
set_nonblocking (self_pipe[0]);
set_nonblocking (self_pipe[1]);
}
/* Close the WebSocket server and clean up. */
void
stop_ws_server (GWSWriter *gwswriter, GWSReader *gwsreader) {
pthread_t writer, reader;
WSServer *server = NULL;
if (!gwsreader || !gwswriter)
return;
if (!(server = gwswriter->server))
return;
pthread_mutex_lock (&gwsreader->mutex);
if ((write (gwsreader->self_pipe[1], "x", 1)) == -1 && errno != EAGAIN)
LOG (("Unable to write to self pipe on pipeout.\n"));
pthread_mutex_unlock (&gwsreader->mutex);
/* if it fails to write, force stop */
pthread_mutex_lock (&gwswriter->mutex);
if ((write (server->self_pipe[1], "x", 1)) == -1 && errno != EAGAIN)
ws_stop (server);
pthread_mutex_unlock (&gwswriter->mutex);
reader = gwsreader->thread;
if (pthread_join (reader, NULL) != 0)
LOG (("Unable to join thread gwsreader: %s\n", strerror (errno)));
writer = gwswriter->thread;
if (pthread_join (writer, NULL) != 0)
LOG (("Unable to join thread gwswriter: %s\n", strerror (errno)));
}
/* Start the WebSocket server and initialize default options. */
static void
start_server (void *ptr_data) {
GWSWriter *writer = (GWSWriter *) ptr_data;
writer->server->onopen = onopen;
#ifdef HAVE_LIBSSL
writer->server->onmessage = onmessage;
#endif
pthread_mutex_lock (&writer->mutex);
set_self_pipe (writer->server->self_pipe);
pthread_mutex_unlock (&writer->mutex);
/* poll(2) will block in here */
ws_start (writer->server);
fprintf (stderr, "Stopping WebSocket server...\n");
ws_stop (writer->server);
}
/* Read and set the WebSocket config options. */
static void
set_ws_opts (void) {
ws_set_config_strict (1);
if (conf.addr)
ws_set_config_host (conf.addr);
if (conf.unix_socket)
ws_set_config_unix_socket (conf.unix_socket);
if (conf.fifo_in)
ws_set_config_pipein (conf.fifo_in);
if (conf.fifo_out)
ws_set_config_pipeout (conf.fifo_out);
if (conf.origin)
ws_set_config_origin (conf.origin);
if (conf.port)
ws_set_config_port (conf.port);
if (conf.sslcert)
ws_set_config_sslcert (conf.sslcert);
if (conf.sslkey)
ws_set_config_sslkey (conf.sslkey);
#ifdef HAVE_LIBSSL
if (conf.ws_auth_secret) {
ws_set_config_auth_secret (conf.ws_auth_secret);
ws_set_config_auth_cb (verify_jwt_token);
}
#endif
}
/* Setup and start the WebSocket threads. */
int
setup_ws_server (GWSWriter *gwswriter, GWSReader *gwsreader) {
int id;
pthread_t *thread;
if (pthread_mutex_init (&gwswriter->mutex, NULL))
FATAL ("Failed init gwswriter mutex");
if (pthread_mutex_init (&gwsreader->mutex, NULL))
FATAL ("Failed init gwsreader mutex");
/* send WS data thread */
thread = &gwswriter->thread;
/* pre-init the websocket server, to ensure the FIFOs are created */
if ((gwswriter->server = ws_init ("0.0.0.0", "7890", set_ws_opts)) == NULL)
FATAL ("Failed init websocket");
id = pthread_create (&(*thread), NULL, (void *) &start_server, gwswriter);
if (id)
FATAL ("Return code from pthread_create(): %d", id);
/* read WS data thread */
thread = &gwsreader->thread;
id = pthread_create (&(*thread), NULL, (void *) &read_client, gwsreader);
if (id)
FATAL ("Return code from pthread_create(): %d", id);
return 0;
}
src/gwsocket.h
+71
View File
@@ -0,0 +1,71 @@
/**
* _______ _______ __ __
* / ____/ | / / ___/____ _____/ /_____ / /_
* / / __ | | /| / /\__ \/ __ \/ ___/ //_/ _ \/ __/
* / /_/ / | |/ |/ /___/ / /_/ / /__/ ,< / __/ /_
* \____/ |__/|__//____/\____/\___/_/|_|\___/\__/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#ifndef GWSOCKET_H_INCLUDED
#define GWSOCKET_H_INCLUDED
#define GW_VERSION "0.1"
#include <pthread.h>
#include "websocket.h"
typedef struct GWSReader_ {
int fd;
int self_pipe[2]; /* self-pipe */
pthread_mutex_t mutex; /* Mutex fifo out */
pthread_t thread; /* Thread fifo in */
WSPacket *packet; /* FIFO data's buffer */
char hdr[HDR_SIZE]; /* FIFO header's buffer */
int hlen; /* header length */
} GWSReader;
typedef struct GWSWriter_ {
int fd;
pthread_mutex_t mutex; /* Mutex fifo in */
pthread_t thread; /* Thread fifo out */
WSServer *server; /* WebSocket server */
} GWSWriter;
GWSReader *new_gwsreader (void);
GWSWriter *new_gwswriter (void);
int broadcast_holder (int fd, const char *buf, int len);
int open_fifoin (void);
int open_fifoout (void);
int read_fifo (GWSReader * gwsreader, void (*f) (int));
int send_holder_to_client (int fd, int listener, const char *buf, int len);
int setup_ws_server (GWSWriter * gwswriter, GWSReader * gwsreader);
void set_ready_state (void);
void set_self_pipe (int *self_pipe);
void stop_ws_server (GWSWriter * gwswriter, GWSReader * gwsreader);
#endif // for #ifndef GWSOCKET_H
src/json.c
+1203
View File
File diff suppressed because it is too large Load Diff
src/json.h
+64
View File
@@ -0,0 +1,64 @@
/**
* ______ ___
* / ____/___ / | _____________ __________
* / / __/ __ \/ /| |/ ___/ ___/ _ \/ ___/ ___/
* / /_/ / /_/ / ___ / /__/ /__/ __(__ |__ )
* \____/\____/_/ |_\___/\___/\___/____/____/
*
* The MIT License (MIT)
* Copyright (c) 2009-2026 Gerardo Orellana <hello @ goaccess.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#if HAVE_CONFIG_H
#include <config.h>
#endif
#ifndef JSON_H_INCLUDED
#define JSON_H_INCLUDED
#define TAB "\t\t\t\t\t\t\t\t\t\t\t"
#define NL "\n\n\n"
#include "parser.h"
typedef struct GJSON_ {
char *buf; /* pointer to buffer */
size_t size; /* size of malloc'd buffer */
size_t offset; /* current write offset */
} GJSON;
char *get_json (GHolder * holder, int escape_html);
void output_json (GHolder * holder, const char *filename);
void set_json_nlines (int nl);
void fpskeyival (FILE * fp, const char *key, int val, int sp, int last);
void fpskeysval (FILE * fp, const char *key, const char *val, int sp, int last);
void fpskeyaval (FILE * fp, const char *key, const char *val, int sp, int last);
void fpclose_arr (FILE * fp, int sp, int last);
void fpclose_obj (FILE * fp, int iisp, int last);
void fpjson (FILE * fp, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
void fpopen_arr_attr (FILE * fp, const char *attr, int sp);
void fpopen_obj_attr (FILE * fp, const char *attr, int sp);
void fpopen_obj (FILE * fp, int iisp);
#endif
src/khash.h
+679
View File
@@ -0,0 +1,679 @@
/* The MIT License
Copyright (c) 2008, 2009, 2011 by Attractive Chaos <attractor@live.co.uk>
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
/*
An example:
#include "khash.h"
KHASH_MAP_INIT_INT(32, char)
int main() {
int ret, is_missing;
khiter_t k;
khash_t(32) *h = kh_init(32);
k = kh_put(32, h, 5, &ret);
kh_value(h, k) = 10;
k = kh_get(32, h, 10);
is_missing = (k == kh_end(h));
k = kh_get(32, h, 5);
kh_del(32, h, k);
for (k = kh_begin(h); k != kh_end(h); ++k)
if (kh_exist(h, k)) kh_value(h, k) = 1;
kh_destroy(32, h);
return 0;
}
*/
/*
2013-05-02 (0.2.8):
* Use quadratic probing. When the capacity is power of 2, stepping function
i*(i+1)/2 guarantees to traverse each bucket. It is better than double
hashing on cache performance and is more robust than linear probing.
In theory, double hashing should be more robust than quadratic probing.
However, my implementation is probably not for large hash tables, because
the second hash function is closely tied to the first hash function,
which reduce the effectiveness of double hashing.
Reference: http://research.cs.vt.edu/AVresearch/hashing/quadratic.php
2011-12-29 (0.2.7):
* Minor code clean up; no actual effect.
2011-09-16 (0.2.6):
* The capacity is a power of 2. This seems to dramatically improve the
speed for simple keys. Thank Zilong Tan for the suggestion. Reference:
- http://code.google.com/p/ulib/
- http://nothings.org/computer/judy/
* Allow to optionally use linear probing which usually has better
performance for random input. Double hashing is still the default as it
is more robust to certain non-random input.
* Added Wang's integer hash function (not used by default). This hash
function is more robust to certain non-random input.
2011-02-14 (0.2.5):
* Allow to declare global functions.
2009-09-26 (0.2.4):
* Improve portability
2008-09-19 (0.2.3):
* Corrected the example
* Improved interfaces
2008-09-11 (0.2.2):
* Improved speed a little in kh_put()
2008-09-10 (0.2.1):
* Added kh_clear()
* Fixed a compiling error
2008-09-02 (0.2.0):
* Changed to token concatenation which increases flexibility.
2008-08-31 (0.1.2):
* Fixed a bug in kh_get(), which has not been tested previously.
2008-08-31 (0.1.1):
* Added destructor
*/
#ifndef __AC_KHASH_H
#define __AC_KHASH_H
/*!
@header
Generic hash table library.
*/
#define AC_VERSION_KHASH_H "0.2.8"
#include <stdlib.h>
#include <string.h>
#include <limits.h>
/* compiler specific configuration */
#if UINT_MAX == 0xffffffffu
typedef unsigned int khint32_t;
#elif ULONG_MAX == 0xffffffffu
typedef unsigned long khint32_t;
#endif
#if ULONG_MAX == ULLONG_MAX
typedef unsigned long khint64_t;
#else
typedef unsigned long long khint64_t;
#endif
#ifndef kh_inline
#ifdef _MSC_VER
#define kh_inline __inline
#else
#define kh_inline inline
#endif
#endif /* kh_inline */
#ifndef klib_unused
#if (defined __clang__ && __clang_major__ >= 3) || (defined __GNUC__ && __GNUC__ >= 3)
#define klib_unused __attribute__ ((__unused__))
#else
#define klib_unused
#endif
#endif /* klib_unused */
typedef khint32_t khint_t;
typedef khint_t khiter_t;
#define __ac_isempty(flag, i) ((flag[i>>4]>>((i&0xfU)<<1))&2)
#define __ac_isdel(flag, i) ((flag[i>>4]>>((i&0xfU)<<1))&1)
#define __ac_iseither(flag, i) ((flag[i>>4]>>((i&0xfU)<<1))&3)
#define __ac_set_isdel_false(flag, i) (flag[i>>4]&=~(1ul<<((i&0xfU)<<1)))
#define __ac_set_isempty_false(flag, i) (flag[i>>4]&=~(2ul<<((i&0xfU)<<1)))
#define __ac_set_isboth_false(flag, i) (flag[i>>4]&=~(3ul<<((i&0xfU)<<1)))
#define __ac_set_isdel_true(flag, i) (flag[i>>4]|=1ul<<((i&0xfU)<<1))
#define __ac_fsize(m) ((m) < 16? 1 : (m)>>4)
#ifndef kroundup32
#define kroundup32(x) (--(x), (x)|=(x)>>1, (x)|=(x)>>2, (x)|=(x)>>4, (x)|=(x)>>8, (x)|=(x)>>16, ++(x))
#endif
#ifndef kcalloc
#define kcalloc(N,Z) calloc(N,Z)
#endif
#ifndef kmalloc
#define kmalloc(Z) malloc(Z)
#endif
#ifndef krealloc
#define krealloc(P,Z) realloc(P,Z)
#endif
#ifndef kfree
#define kfree(P) free(P)
#endif
static const double __ac_HASH_UPPER = 0.77;
#define __KHASH_TYPE(name, khkey_t, khval_t) \
typedef struct kh_##name##_s { \
khint_t n_buckets, size, n_occupied, upper_bound; \
khint32_t *flags; \
khkey_t *keys; \
khval_t *vals; \
} kh_##name##_t;
#define __KHASH_PROTOTYPES(name, khkey_t, khval_t) \
extern kh_##name##_t *kh_init_##name(void); \
extern void kh_destroy_##name(kh_##name##_t *h); \
extern void kh_clear_##name(kh_##name##_t *h); \
extern khint_t kh_get_##name(const kh_##name##_t *h, khkey_t key); \
extern int kh_resize_##name(kh_##name##_t *h, khint_t new_n_buckets); \
extern khint_t kh_put_##name(kh_##name##_t *h, khkey_t key, int *ret); \
extern void kh_del_##name(kh_##name##_t *h, khint_t x); \
#define __KHASH_IMPL(name, SCOPE, khkey_t, khval_t, kh_is_map, __hash_func, __hash_equal) \
SCOPE kh_##name##_t *kh_init_##name(void) { \
return (kh_##name##_t*) kcalloc(1, sizeof(kh_##name##_t)); \
} \
SCOPE void kh_destroy_##name(kh_##name##_t *h) \
{ \
if (h) { \
kfree ((void *) h->keys); \
kfree (h->flags); \
kfree ((void *) h->vals); \
kfree (h); \
} \
} \
SCOPE void kh_clear_##name(kh_##name##_t *h) \
{ \
if (h && h->flags) { \
memset (h->flags, 0xaa, __ac_fsize (h->n_buckets) * sizeof (khint32_t)); \
h->size = h->n_occupied = 0; \
} \
} \
SCOPE khint_t kh_get_##name(const kh_##name##_t *h, khkey_t key) \
{ \
if (h->n_buckets) { \
khint_t k, i, last, mask, step = 0; \
mask = h->n_buckets - 1; \
k = __hash_func (key); \
i = k & mask; \
last = i; \
while (!__ac_isempty (h->flags, i) && \
(__ac_isdel (h->flags, i) || !__hash_equal (h->keys[i], key))) { \
i = (i + (++step)) & mask; \
if (i == last) \
return h->n_buckets; \
} \
return __ac_iseither (h->flags, i) ? h->n_buckets : i; \
} else \
return 0; \
} \
SCOPE int kh_resize_##name(kh_##name##_t *h, khint_t new_n_buckets) \
{ \
/* This function uses 0.25*n_buckets bytes of working space instead of */ \
/* [sizeof(key_t+val_t)+.25]*n_buckets. */ \
khint32_t *new_flags = 0; \
khint_t j = 1; \
{ \
kroundup32 (new_n_buckets); \
if (new_n_buckets < 4) \
new_n_buckets = 4; \
if (h->size >= (khint_t) (new_n_buckets * __ac_HASH_UPPER + 0.5)) \
j = 0; /* requested size is too small */ \
else { /* hash table size to be changed (shrink or expand); rehash */ \
new_flags = (khint32_t *) kmalloc (__ac_fsize (new_n_buckets) * sizeof (khint32_t)); \
if (!new_flags) \
return -1; \
memset (new_flags, 0xaa, __ac_fsize (new_n_buckets) * sizeof (khint32_t)); \
if (h->n_buckets < new_n_buckets) { /* expand */ \
khkey_t *new_keys = (khkey_t *) krealloc ((void *) h->keys, new_n_buckets * sizeof (khkey_t)); \
if (!new_keys) { \
kfree (new_flags); \
return -1; \
} \
h->keys = new_keys; \
if (kh_is_map) { \
khval_t *new_vals = (khval_t *) krealloc ((void *) h->vals, new_n_buckets * sizeof (khval_t)); \
if (!new_vals) { \
kfree (new_flags); \
return -1; \
} \
h->vals = new_vals; \
} \
} /* otherwise shrink */ \
} \
} \
if (j) { /* rehashing is needed */ \
for (j = 0; j != h->n_buckets; ++j) { \
if (__ac_iseither (h->flags, j) == 0) { \
khkey_t key = h->keys[j]; \
khval_t val; \
khint_t new_mask; \
new_mask = new_n_buckets - 1; \
if (kh_is_map) \
val = h->vals[j]; \
__ac_set_isdel_true (h->flags, j); \
while (1) { /* kick-out process; sort of like in Cuckoo hashing */ \
khint_t k, i, step = 0; \
k = __hash_func (key); \
i = k & new_mask; \
while (!__ac_isempty (new_flags, i)) \
i = (i + (++step)) & new_mask; \
__ac_set_isempty_false (new_flags, i); \
if (i < h->n_buckets && __ac_iseither (h->flags, i) == 0) { /* kick out the existing element */ \
{ \
khkey_t tmp = h->keys[i]; \
h->keys[i] = key; \
key = tmp; \
} \
if (kh_is_map) { \
khval_t tmp = h->vals[i]; \
h->vals[i] = val; \
val = tmp; \
} \
__ac_set_isdel_true (h->flags, i); /* mark it as deleted in the old hash table */ \
} else { /* write the element and jump out of the loop */ \
h->keys[i] = key; \
if (kh_is_map) \
h->vals[i] = val; \
break; \
} \
} \
} \
} \
if (h->n_buckets > new_n_buckets) { /* shrink the hash table */ \
h->keys = (khkey_t *) krealloc ((void *) h->keys, new_n_buckets * sizeof (khkey_t)); \
if (kh_is_map) \
h->vals = (khval_t *) krealloc ((void *) h->vals, new_n_buckets * sizeof (khval_t)); \
} \
kfree (h->flags); /* free the working space */ \
h->flags = new_flags; \
h->n_buckets = new_n_buckets; \
h->n_occupied = h->size; \
h->upper_bound = (khint_t) (h->n_buckets * __ac_HASH_UPPER + 0.5); \
} \
return 0; \
} \
SCOPE khint_t kh_put_##name(kh_##name##_t *h, khkey_t key, int *ret) \
{ \
khint_t x; \
if (h->n_occupied >= h->upper_bound) { /* update the hash table */ \
if (h->n_buckets > (h->size << 1)) { \
if (kh_resize_##name (h, h->n_buckets - 1) < 0) { /* clear "deleted" elements */ \
*ret = -1; \
return h->n_buckets; \
} \
} else if (kh_resize_##name (h, h->n_buckets + 1) < 0) { /* expand the hash table */ \
*ret = -1; \
return h->n_buckets; \
} \
} /* TODO: to implement automatically shrinking; resize() already support shrinking */ \
{ \
khint_t k, i, site, last, mask = h->n_buckets - 1, step = 0; \
x = site = h->n_buckets; \
k = __hash_func (key); \
i = k & mask; \
if (__ac_isempty (h->flags, i)) \
x = i; /* for speed up */ \
else { \
last = i; \
while (!__ac_isempty (h->flags, i) && (__ac_isdel (h->flags, i) || !__hash_equal (h->keys[i], key))) { \
if (__ac_isdel (h->flags, i)) \
site = i; \
i = (i + (++step)) & mask; \
if (i == last) { \
x = site; \
break; \
} \
} \
if (x == h->n_buckets) { \
if (__ac_isempty (h->flags, i) && site != h->n_buckets) \
x = site; \
else \
x = i; \
} \
} \
} \
if (__ac_isempty (h->flags, x)) { /* not present at all */ \
h->keys[x] = key; \
__ac_set_isboth_false (h->flags, x); \
++h->size; \
++h->n_occupied; \
*ret = 1; \
} else if (__ac_isdel (h->flags, x)) { /* deleted */ \
h->keys[x] = key; \
__ac_set_isboth_false (h->flags, x); \
++h->size; \
*ret = 2; \
} else \
*ret = 0; /* Don't touch h->keys[x] if present and not deleted */ \
return x; \
} \
SCOPE void kh_del_##name(kh_##name##_t *h, khint_t x) \
{ \
if (x != h->n_buckets && !__ac_iseither (h->flags, x)) { \
__ac_set_isdel_true (h->flags, x); \
--h->size; \
} \
} \
#define KHASH_DECLARE(name, khkey_t, khval_t) \
__KHASH_TYPE(name, khkey_t, khval_t) \
__KHASH_PROTOTYPES(name, khkey_t, khval_t)
#define KHASH_INIT2(name, SCOPE, khkey_t, khval_t, kh_is_map, __hash_func, __hash_equal) \
__KHASH_TYPE(name, khkey_t, khval_t) \
__KHASH_IMPL(name, SCOPE, khkey_t, khval_t, kh_is_map, __hash_func, __hash_equal)
#define KHASH_INIT(name, khkey_t, khval_t, kh_is_map, __hash_func, __hash_equal) \
KHASH_INIT2(name, static kh_inline klib_unused, khkey_t, khval_t, kh_is_map, __hash_func, __hash_equal) \
/* --- BEGIN OF HASH FUNCTIONS --- */
/*! @function
@abstract Integer hash function
@param key The integer [khint32_t]
@return The hash value [khint_t]
*/
#define kh_int_hash_func(key) (khint32_t)(key)
/*! @function
@abstract Integer comparison function
*/
#define kh_int_hash_equal(a, b) ((a) == (b))
/*! @function
@abstract 64-bit integer hash function
@param key The integer [khint64_t]
@return The hash value [khint_t]
*/
#define kh_int64_hash_func(key) (khint32_t)((key)>>33^(key)^(key)<<11)
/*! @function
@abstract 64-bit integer comparison function
*/
#define kh_int64_hash_equal(a, b) ((a) == (b))
/*! @function
@abstract const char* hash function
@param s Pointer to a null terminated string
@return The hash value
*/
#if defined(__clang__) && defined(__clang_major__) && (__clang_major__ >= 4)
__attribute__((no_sanitize ("unsigned-integer-overflow")))
#if (__clang_major__ >= 12)
__attribute__((no_sanitize ("unsigned-shift-base")))
#endif
#endif
static kh_inline khint_t
__ac_X31_hash_string (const char *s) {
khint_t h = (khint_t) * s;
if (h)
for (++s; *s; ++s)
h = (h << 5) - h + (khint_t) * s;
return h;
}
/*! @function
@abstract Another interface to const char* hash function
@param key Pointer to a null terminated string [const char*]
@return The hash value [khint_t]
*/
#define kh_str_hash_func(key) __ac_X31_hash_string(key)
/*! @function
@abstract Const char* comparison function
*/
#define kh_str_hash_equal(a, b) (strcmp(a, b) == 0)
static kh_inline khint_t
__ac_Wang_hash (khint_t key) {
key += ~(key << 15);
key ^= (key >> 10);
key += (key << 3);
key ^= (key >> 6);
key += ~(key << 11);
key ^= (key >> 16);
return key;
}
#define kh_int_hash_func2(k) __ac_Wang_hash((khint_t)key)
/* --- END OF HASH FUNCTIONS --- */
/* Other convenient macros... */
/*!
@abstract Type of the hash table.
@param name Name of the hash table [symbol]
*/
#define khash_t(name) kh_##name##_t
/*! @function
@abstract Initiate a hash table.
@param name Name of the hash table [symbol]
@return Pointer to the hash table [khash_t(name)*]
*/
#define kh_init(name) kh_init_##name()
/*! @function
@abstract Destroy a hash table.
@param name Name of the hash table [symbol]
@param h Pointer to the hash table [khash_t(name)*]
*/
#define kh_destroy(name, h) kh_destroy_##name(h)
/*! @function
@abstract Reset a hash table without deallocating memory.
@param name Name of the hash table [symbol]
@param h Pointer to the hash table [khash_t(name)*]
*/
#define kh_clear(name, h) kh_clear_##name(h)
/*! @function
@abstract Resize a hash table.
@param name Name of the hash table [symbol]
@param h Pointer to the hash table [khash_t(name)*]
@param s New size [khint_t]
*/
#define kh_resize(name, h, s) kh_resize_##name(h, s)
/*! @function
@abstract Insert a key to the hash table.
@param name Name of the hash table [symbol]
@param h Pointer to the hash table [khash_t(name)*]
@param k Key [type of keys]
@param r Extra return code: -1 if the operation failed;
0 if the key is present in the hash table;
1 if the bucket is empty (never used); 2 if the element in
the bucket has been deleted [int*]
@return Iterator to the inserted element [khint_t]
*/
#define kh_put(name, h, k, r) kh_put_##name(h, k, r)
/*! @function
@abstract Retrieve a key from the hash table.
@param name Name of the hash table [symbol]
@param h Pointer to the hash table [khash_t(name)*]
@param k Key [type of keys]
@return Iterator to the found element, or kh_end(h) if the element is absent [khint_t]
*/
#define kh_get(name, h, k) kh_get_##name(h, k)
/*! @function
@abstract Remove a key from the hash table.
@param name Name of the hash table [symbol]
@param h Pointer to the hash table [khash_t(name)*]
@param k Iterator to the element to be deleted [khint_t]
*/
#define kh_del(name, h, k) kh_del_##name(h, k)
/*! @function
@abstract Test whether a bucket contains data.
@param h Pointer to the hash table [khash_t(name)*]
@param x Iterator to the bucket [khint_t]
@return 1 if containing data; 0 otherwise [int]
*/
#define kh_exist(h, x) (!__ac_iseither((h)->flags, (x)))
/*! @function
@abstract Get key given an iterator
@param h Pointer to the hash table [khash_t(name)*]
@param x Iterator to the bucket [khint_t]
@return Key [type of keys]
*/
#define kh_key(h, x) ((h)->keys[x])
/*! @function
@abstract Get value given an iterator
@param h Pointer to the hash table [khash_t(name)*]
@param x Iterator to the bucket [khint_t]
@return Value [type of values]
@discussion For hash sets, calling this results in segfault.
*/
#define kh_val(h, x) ((h)->vals[x])
/*! @function
@abstract Alias of kh_val()
*/
#define kh_value(h, x) ((h)->vals[x])
/*! @function
@abstract Get the start iterator
@param h Pointer to the hash table [khash_t(name)*]
@return The start iterator [khint_t]
*/
#define kh_begin(h) (khint_t)(0)
/*! @function
@abstract Get the end iterator
@param h Pointer to the hash table [khash_t(name)*]
@return The end iterator [khint_t]
*/
#define kh_end(h) ((h)->n_buckets)
/*! @function
@abstract Get the number of elements in the hash table
@param h Pointer to the hash table [khash_t(name)*]
@return Number of elements in the hash table [khint_t]
*/
#define kh_size(h) ((h)->size)
/*! @function
@abstract Get the number of buckets in the hash table
@param h Pointer to the hash table [khash_t(name)*]
@return Number of buckets in the hash table [khint_t]
*/
#define kh_n_buckets(h) ((h)->n_buckets)
/*! @function
@abstract Iterate over the entries in the hash table
@param h Pointer to the hash table [khash_t(name)*]
@param kvar Variable to which key will be assigned
@param vvar Variable to which value will be assigned
@param code Block of code to execute
*/
#define kh_foreach(h, kvar, vvar, code) { khint_t __i; \
for (__i = kh_begin(h); __i != kh_end(h); ++__i) { \
if (!kh_exist(h,__i)) continue; \
(kvar) = kh_key(h,__i); \
(vvar) = kh_val(h,__i); \
code; \
} }
/*! @function
@abstract Iterate over the values in the hash table
@param h Pointer to the hash table [khash_t(name)*]
@param vvar Variable to which value will be assigned
@param code Block of code to execute
*/
#define kh_foreach_value(h, vvar, code) { khint_t __i; \
for (__i = kh_begin (h); __i != kh_end (h); ++__i) { \
if (!kh_exist (h, __i)) \
continue; \
(vvar) = kh_val (h, __i); \
code; \
} }
/* More convenient interfaces */
/*! @function
@abstract Instantiate a hash set containing integer keys
@param name Name of the hash table [symbol]
*/
#define KHASH_SET_INIT_INT(name) \
KHASH_INIT(name, khint32_t, char, 0, kh_int_hash_func, kh_int_hash_equal)
/*! @function
@abstract Instantiate a hash map containing integer keys
@param name Name of the hash table [symbol]
@param khval_t Type of values [type]
*/
#define KHASH_MAP_INIT_INT(name, khval_t) \
KHASH_INIT(name, khint32_t, khval_t, 1, kh_int_hash_func, kh_int_hash_equal)
/*! @function
@abstract Instantiate a hash map containing 64-bit integer keys
@param name Name of the hash table [symbol]
*/
#define KHASH_SET_INIT_INT64(name) \
KHASH_INIT(name, khint64_t, char, 0, kh_int64_hash_func, kh_int64_hash_equal)
/*! @function
@abstract Instantiate a hash map containing 64-bit integer keys
@param name Name of the hash table [symbol]
@param khval_t Type of values [type]
*/
#define KHASH_MAP_INIT_INT64(name, khval_t) \
KHASH_INIT(name, khint64_t, khval_t, 1, kh_int64_hash_func, kh_int64_hash_equal)
typedef const char *kh_cstr_t;
/*! @function
@abstract Instantiate a hash map containing const char* keys
@param name Name of the hash table [symbol]
*/
#define KHASH_SET_INIT_STR(name) \
KHASH_INIT(name, kh_cstr_t, char, 0, kh_str_hash_func, kh_str_hash_equal)
/*! @function
@abstract Instantiate a hash map containing const char* keys
@param name Name of the hash table [symbol]
@param khval_t Type of values [type]
*/
#define KHASH_MAP_INIT_STR(name, khval_t) \
KHASH_INIT(name, kh_cstr_t, khval_t, 1, kh_str_hash_func, kh_str_hash_equal)
#endif /* __AC_KHASH_H */

Some files were not shown because too many files have changed in this diff Show More