Skip to main content

Raymii.org Raymii.org Logo

Quis custodiet ipsos custodes?
Home | About | All pages | Cluster Status | RSS Feed | Gopher

Firefox History stats with Bash

Published: 25-09-2016 | Author: Remy van Elst | Text only version of this article


❗ This post is over four years old. It may no longer be up to date. Opinions may have changed.

This is a small script to gather some statistics from your Firefox history. First we use sqlite3 to parse the Firefox history database and get the last three months, then we remove all the IP addresses and port numbers and finally we sort and count it.

Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.

You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $100 credit for 60 days.

#!/bin/bash
if [[ ! -d ${HOME}/.www ]]; then
  mkdir ${HOME}/.www/
fi

cp "$(find "${HOME}/.mozilla/firefox/" -name "places.sqlite" | head -n 1)" "${HOME}/.www/places.sqlite"
sqlite3 "${HOME}/.www/places.sqlite" "SELECT url FROM moz_places, moz_historyvisits \
                       WHERE moz_places.id = moz_historyvisits.place_id \
                             and visit_date > strftime('%s','now','-3 month')*1000000 ORDER by \
                       visit_date;"  > "${HOME}/.www/urls-unsorted"
sort -u "${HOME}/.www/urls-unsorted" > "${HOME}/.www/urls"

awk -F/ '{print $3}' "${HOME}/.www/urls" | grep -v -E -e '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' -e ':.*' -e '^$' | sed -e 's/www\.//g' |sort | uniq -c | sort -n 

Example output:

    383 lowendtalk.com
    534 reddit.com
    569 google.com
    574 github.com
    792 encrypted.google.com
    973 i.imgur.com
   1458 next.duckduckgo.com
   3009 duckduckgo.com
   6459 wiki.int
  12934 management.int

If you just want HTTPS domains, or just HTTP you can use awk to make that happen:

awk -F/ '$1 == "https:" {print $3}' "${HOME}/.www/urls" | grep -v -E -e '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | sort | uniq -c | sort
Tags: bash , firefox , history , snippets , sqlite , ubuntu