Raymii.org
Quis custodiet ipsos custodes?Home | About | All pages | Cluster Status | RSS Feed
Firefox History stats with Bash
Published: 25-09-2016 | Author: Remy van Elst | Text only version of this article
❗ This post is over seven years old. It may no longer be up to date. Opinions may have changed.
This is a small script to gather some statistics from your Firefox history. First we use sqlite3 to parse the Firefox history database and get the last three months, then we remove all the IP addresses and port numbers and finally we sort and count it.
Recently I removed all Google Ads from this site due to their invasive tracking, as well as Google Analytics. Please, if you found this content useful, consider a small donation using any of the options below:
I'm developing an open source monitoring app called Leaf Node Monitoring, for windows, linux & android. Go check it out!
Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.
You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $100 credit for 60 days.
#!/bin/bash
if [[ ! -d ${HOME}/.www ]]; then
mkdir ${HOME}/.www/
fi
cp "$(find "${HOME}/.mozilla/firefox/" -name "places.sqlite" | head -n 1)" "${HOME}/.www/places.sqlite"
sqlite3 "${HOME}/.www/places.sqlite" "SELECT url FROM moz_places, moz_historyvisits \
WHERE moz_places.id = moz_historyvisits.place_id \
and visit_date > strftime('%s','now','-3 month')*1000000 ORDER by \
visit_date;" > "${HOME}/.www/urls-unsorted"
sort -u "${HOME}/.www/urls-unsorted" > "${HOME}/.www/urls"
awk -F/ '{print $3}' "${HOME}/.www/urls" | grep -v -E -e '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' -e ':.*' -e '^$' | sed -e 's/www\.//g' |sort | uniq -c | sort -n
Example output:
383 lowendtalk.com
534 reddit.com
569 google.com
574 github.com
792 encrypted.google.com
973 i.imgur.com
1458 next.duckduckgo.com
3009 duckduckgo.com
6459 wiki.int
12934 management.int
If you just want HTTPS domains, or just HTTP you can use awk to make that happen:
awk -F/ '$1 == "https:" {print $3}' "${HOME}/.www/urls" | grep -v -E -e '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | sort | uniq -c | sort