Another web scraping example.
A nice script to do simple web scraping and how to read json with the command-line.
web
Easily scrape data from a table with htmlq on Linux.
How to scrape content from a web site and properly format the resulting output.
Easy web scraping with Powershell. This is very simple.
How to do simple web scraping on Linux and Windows easily. This is very useful.
Another example of web scraping using the htmlq library for Python.
A couple of very useful web scraping examples to try with Python.
Read the news headlines from the Daily Telegraph using Powershell.
A few web scraping examples for Powershell. How to get news headlines from a website.
Lenovo BIOS simulator. This could be very useful indeed.
A very useful BIOS simulator page I found, this is for Lenovo machines.
Some very useful userscripts for use with Tampermonkey.
There are a lot of very useful userscripts on the Internet, these are just a few of the most useful ones. No YouTube Volume Normalization. https://gist.github.com/abec2304/2782f4fc47f9d010dfaab00f25e69c8a. This userscript will perform volume normalization on Youtube videos, this will help out when you are using Youtube for Vevo. Web Comic reader. https://github.com/ameboide/webcomic_reader/blob/master/webcomic_reader.user.js. Pretty good for reading webcomics … Read more
Parsing Apache logs with awk is a lot of fun and easy too.
Parsing Apache logs is a lot of fun with awk on Linux. This can be very interesting, to see all of the bots and other visitors you get, there are so many crawlers accessing your website and indexing all of your pages. But this can be used to find unwanted leaching of bandwidth as well, … Read more
The old web is vanishing.
The many websites I used to enjoy like forums are vanishing and are not all being archived. This is a pity, especially if the forums had various solutions on them that another forum does not have, then the information is lost forever. But this is the evolution of the Internet. The sharing of information is … Read more
Nice web scraper to get a listing of all 4chan threads on a certain board.
This is a nice web scraper that will read a 4chan board and return a listing of all threads on that board page. This could be very useful code to expand into a useful script. #!/usr/bin/env bash set -e links=( $( wget "$@" -qo /dev/null -SO – | grep -oE ‘</span><a href=\"thread\/[0-9]+\"’ | … Read more
Find insecure web servers easily with these Google Dorks.
These easy to use Google Searches make finding insecure web servers very easy. This one finds servers with publicly accessible pages that show web site status. They even have a complete listing of all ports and whether they are open or not! intitle:"web server status" SSH Telnetintitle:"web server status" SSH Telnet Why would a web … Read more
How to filter the server name from SSL client hello packets with Wireshark.
SSL packets sent to and fro from your machine can contain a lot of useful information. I am going to show how to see what websites are visited on a network. Firstly, load Wireshark and then listen to traffic on your machine. Then visit a few different websites with Firefox or Chrome. Use this filter … Read more
Amazon web services. The perfect alternative to a locally hosted server solution.
Amazon Web Services is the perfect solution for hosting a server outside your organization. This allows you to create a cloud hosted database server that takes the strain off your local server infrastructure using Microsoft SQL server 2012 or a Linux based solution in a cloud hosted server instance. There is a free trial that … Read more
How to scan for vulnerable ports on a host with the Kali Linux distribution.
Scanning for vulnerable ports is easy with the Kali Linux distribution. Here is a simple port scan of my Windows Server 2012 R2 laptop host OS using Kali Linux. homer@kali:~$ sudo nmap 192.168.1.6 Starting Nmap 6.46 ( http://nmap.org ) at 2014-05-28 20:44 EST Nmap scan report for 192.168.1.6 Host is up (1.0s latency). Not … Read more
How to get the geographical location of an IP address.
How to get the geographical location of an IP address. This simple command will get this for you. [jason@darknet:~] curl ipinfo.io/203.113.124.148 ; echo "" { "ip": "203.113.124.148", "hostname": "No Hostname", "city": "", "region": "", "country": "TH", "loc": "13.7500,100.4667", "org": "AS9737 TOT Public Company Limited" }[jason@darknet:~] curl ipinfo.io/203.113.124.148 ; echo "" { "ip": "203.113.124.148", "hostname": "No … Read more
Thoughts on the new Apple SSL vulnerability.
The new Apple SSL vulnerability is a concerning bug in the Apple Macintosh iOS and desktop operating systems. This allows SSL spoofing as it will allow a faked SSL vulnerability to pass as a real one. Apparently it uses ports 1266 & 1267, if you block these with your firewall, this should alleviate this threat … Read more
Watch computer booting sequence in your web browser. This is a very cool web site I found.
http://www.therestartpage.com/#. This is a website that has shutdown dialogs for a host of operating systems. Click restart on one of them and you can watch the restart animation and/or POST for that OS. This is one very cool website. The ability to see what the boot animation for Windows `95 and 2000 looked like really … Read more
Why is Responsive Web Design Important?
The onslaught of technological innovations over the last decade posed an enormous challenge to web developers in adapting to the various devices, platforms and browsers. Every day we see a new smart phone, tablet and notebooks being launched and this mad invasion of Technology is only meant to grow bigger by the day. Why Responsive … Read more
First Hypertext page ever created resurrected in 2013!
http://info.cern.ch/hypertext/WWW/TheProject.html. This is the first Hypertext page ever created; the page that was the first to use hyperlinks and gave us a connected and more informed future. This is a page using some very old HTML 1.0 code with<DD> tags that I have not seen for a very long time. They seem to have gone … Read more