Skip to main content

Raymii.org Logo (IEC resistor symbol) logo

Quis custodiet ipsos custodes?
Home | About | All pages | RSS Feed | Gopher

Firefox History stats with Bash

Published: 25-09-2016 | Author: Remy van Elst | Text only version of this article


Table of Contents


This is a small script to gather some statistics from your Firefox history. First we use sqlite3 to parse the Firefox history database and get the last three months, then we remove all the IP addresses and port numbers and finally we sort and count it.

If you like this article, consider sponsoring me by trying out a Digital Ocean VPS. With this link you'll get $100 credit for 60 days). (referral link)

#!/bin/bash
if [[ ! -d ${HOME}/.www ]]; then
  mkdir ${HOME}/.www/
fi

cp "$(find "${HOME}/.mozilla/firefox/" -name "places.sqlite" | head -n 1)" "${HOME}/.www/places.sqlite"
sqlite3 "${HOME}/.www/places.sqlite" "SELECT url FROM moz_places, moz_historyvisits \
                       WHERE moz_places.id = moz_historyvisits.place_id \
                             and visit_date > strftime('%s','now','-3 month')*1000000 ORDER by \
                       visit_date;"  > "${HOME}/.www/urls-unsorted"
sort -u "${HOME}/.www/urls-unsorted" > "${HOME}/.www/urls"

awk -F/ '{print $3}' "${HOME}/.www/urls" | grep -v -E -e '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' -e ':.*' -e '^$' | sed -e 's/www\.//g' |sort | uniq -c | sort -n 

Example output:

    383 lowendtalk.com
    534 reddit.com
    569 google.com
    574 github.com
    792 encrypted.google.com
    973 i.imgur.com
   1458 next.duckduckgo.com
   3009 duckduckgo.com
   6459 wiki.int
  12934 management.int

If you just want HTTPS domains, or just HTTP you can use awk to make that happen:

awk -F/ '$1 == "https:" {print $3}' "${HOME}/.www/urls" | grep -v -E -e '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | sort | uniq -c | sort
Tags: bash , firefox , history , snippets , sqlite , ubuntu