These are the news items I've curated in my monitoring of the API space that have some relevance to the terms of service conversation and I wanted to include in my research. I'm using all of these links to better understand how the space is testing their APIs, going beyond just monitoring and understand the details of each request and response.07 Aug 2017
I was reading about the difficulties the City of New York was having when it comes to migrating off of the Palantir platfom, while also reading about the latest cybersecurity drama involving ransomware. I’m spending a lot of time studying cybersecurity lately, partly because they involve APIs, but mostly because it is something that is impacting every aspect of our lives, including our democracy, education, and healthcare. One thing I notice on the cybersecurity stage, is that everything is a much more extreme, intense, representation of what is going on in the mainstream tech industry.
Ransomware is software that gets installed on your desktop or servers and locks up all your data until you pay the software developer (implementor) a ransom. Ransomware is just a much faster moving version of what many of us in the software industry call vendor lock-in. This is what you are seeing with Palantir, and the City of New York. What tech companies do is get you to install their software on your desktop or servers, or convince you to upload all your data into the cloud, and use their software. This is business 101 in the tech industry. You either develop cloud-based software, something that runs on-premise, or you are a mix of both. Ideally, your customers become dependent on you, and they keep paying your monthly, quarterly, or annual subscriptions (cough cough ransom).
Here is where the crafty API part of the scheme comes in. Software providers can also make APIs that allow your desktop and server to integrate with their cloud solutions, allowing for much deeper integration of data, content, and algorithms. The presence of APIs SHOULD also mean that you can more easily get your data, content, and algorithms back, or have kept in sync the whole time, so that when you are ready to move on, you don’t have a problem getting your data and content back. The problem is, that APIs “CAN” enable this, but in many situations providers do not actually give you complete access to your data, content, or algorithms via API, and enable the true data portability and sync features you need to continue doing business without them.
This is vendor lock-in. It is a much friendlier, slower moving version of ransomware. As a software vendor you want your software to be baked in to a customers operations so they are dependent on you. How aggressively you pursue this, and how much you limit data portability, and interoperability, dictates whether you are just doing business as usual, or engaging in vendor lock-in. One thing I’m hopeful for in all of this, are the vendors who see transparency, observability, interoperability, and portability of not just the technical, but also the business and politics of delivering technology as a legitimate competitive advantage. This means that they will always be able to out maneuver, and stay ahead of software vendors who practice vendor lock-in and ransomware, whether of the slow or fast moving variety.
I came across this valuable list of threat intelligence resources and think that the section on information sources should be aggregated and provided as a single threat intelligence API. When I come across valuable information repos like this my first impulse is to go through them, standardize and upload as JSON and YAML to Github, making all of this data forkable, and available via an API.
Of course if I responded to every impulse like this I would never get any of my normal work done, and actually pay my bills. A second option for me is to put things out there publicly in hopes that a) someone will pay me to do the work, or b) someone else who has more time, and the rent paid will tackle the work. With this in mind, this list of sources should be standardized, and publish to Github and as an API:
|Alexa Top 1 Million sites||Probable Whitelist of the top 1 Million sites from Amazon(Alexa).|
|APT Groups and Operations||A spreadsheet containing information and intelligence about APT groups, operations and tactics.|
|AutoShun||A public service offering at most 2000 malicious IPs and some more resources.|
|BGP Ranking||Ranking of ASNs having the most malicious content.|
|Botnet Tracker||Tracks several active botnets.|
|BruteForceBlocker||BruteForceBlocker is a perl script that monitors a server's sshd logs and identifies brute force attacks, which it then uses to automatically configure firewall blocking rules and submit those IPs back to the project site, http://danger.rulez.sk/projects/bruteforceblocker/blist.php.|
|C&C Tracker||A feed of known, active and non-sinkholed C&C IP addresses, from Bambenek Consulting.|
|CI Army List||A subset of the commercial CINS Score list, focused on poorly rated IPs that are not currently present on other threatlists.|
|Cisco Umbrella||Probable Whitelist of the top 1 million sites resolved by Cisco Umbrella (was OpenDNS).|
|Critical Stack Intel||The free threat intelligence parsed and aggregated by Critical Stack is ready for use in any Bro production system. You can specify which feeds you trust and want to ingest.|
|C1fApp||C1fApp is a threat feed aggregation application, providing a single feed, both Open Source and private. Provides statistics dashboard, open API for search and is been running for a few years now. Searches are on historical data.|
|Cymon||Cymon is an aggregator of indicators from multiple sources with history, so you have a single interface to multiple threat feeds. It also provides an API to search a database along with a pretty web interface.|
|Deepviz Threat Intel||Deepviz offers a sandbox for analyzing malware and has an API available with threat intelligence harvested from the sandbox.|
|Emerging Threats Firewall Rules||A collection of rules for several types of firewalls, including iptables, PF and PIX.|
|Emerging Threats IDS Rules||A collection of Snort and Suricata rules files that can be used for alerting or blocking.|
|ExoneraTor||The ExoneraTor service maintains a database of IP addresses that have been part of the Tor network. It answers the question whether there was a Tor relay running on a given IP address on a given date.|
|Exploitalert||Listing of latest exploits released.|
|ZeuS Tracker||The Feodo Tracker abuse.ch tracks the Feodo trojan.|
|FireHOL IP Lists||400+ publicly available IP Feeds analysed to document their evolution, geo-map, age of IPs, retention policy, overlaps. The site focuses on cyber crime (attacks, abuse, malware).|
|FraudGuard||FraudGuard is a service designed to provide an easy way to validate usage by continuously collecting and analyzing real-time internet traffic.|
|Hail a TAXII||Hail a TAXII.com is a repository of Open Source Cyber Threat Intelligence feeds in STIX format. They offer several feeds, including some that are listed here already in a different format, like the Emerging Threats rules and PhishTank feeds.|
|I-Blocklist||I-Blocklist maintains several types of lists containing IP addresses belonging to various categories. Some of these main categories include countries, ISPs and organizations. Other lists include web attacks, TOR, spyware and proxies. Many are free to use, and available in various formats.|
|Majestic Million||Probable Whitelist of the top 1 million web sites, as ranked by Majestic. Sites are ordered by the number of referring subnets. More about the ranking can be found on their blog.|
|MalShare.com||The MalShare Project is a public malware repository that provides researchers free access to samples.|
|MalwareDomains.com||The DNS-BH project creates and maintains a listing of domains that are known to be used to propagate malware and spyware. These can be used for detection as well as prevention (sinkholing DNS requests).|
|Metadefender.com||Metadefender Cloud Threat Intelligence Feeds contains top new malware hash signatures, including MD5, SHA1, and SHA256. These new malicious hashes have been spotted by Metadefender Cloud within the last 24 hours. The feeds are updated daily with newly detected and reported malware to provide actionable and timely threat intelligence.|
|NormShield Services||NormShield Services provide thousands of domain information (including whois information) that potential phishing attacks may come from. Breach and blacklist services also available. There is free sign up for public services for continuous monitoring.|
|OpenBL.org||A feed of IP addresses found to be attempting brute-force logins on services such as SSH, FTP, IMAP and phpMyAdmin and other web applications.|
|OpenPhish Feeds||OpenPhish receives URLs from multiple streams and analyzes them using its proprietary phishing detection algorithms. There are free and commercial offerings available.|
|PhishTank||PhishTank delivers a list of suspected phishing URLs. Their data comes from human reports, but they also ingest external feeds where possible. It's a free service, but registering for an API key is sometimes necessary.|
|Ransomware Tracker||The Ransomware Tracker by abuse.ch tracks and monitors the status of domain names, IP addresses and URLs that are associated with Ransomware, such as Botnet C∓C servers, distribution sites and payment sites.|
|SANS ICS Suspicious Domains||
The Suspicious Domains Threat Lists by SANS ICS tracks suspicious domains. It offers 3 lists categorized as either high, medium or low sensitivity, where the high sensitivity list has fewer false positives, whereas the low sensitivty list with more false positives. There is also an approved whitelist of domains.
Finally, there is a suggested IP blocklist from DShield.
|signature-base||A database of signatures used in other tools by Neo23x0.|
|The Spamhaus project||The Spamhaus Project contains multiple threatlists associated with spam and malware activity.|
|SSL Blacklist||SSL Blacklist (SSLBL) is a project maintained by abuse.ch. The goal is to provide a list of "bad" SSL certificates identified by abuse.ch to be associated with malware or botnet activities. SSLBL relies on SHA1 fingerprints of malicious SSL certificates and offers various blacklists|
|Statvoo Top 1 Million Sites||Probable Whitelist of the top 1 million web sites, as ranked by Statvoo.|
|Strongarm, by Percipient Networks||Strongarm is a DNS blackhole that takes action on indicators of compromise by blocking malware command and control. Strongarm aggregates free indicator feeds, integrates with commercial feeds, utilizes Percipient's IOC feeds, and operates DNS resolvers and APIs for you to use to protect your network and business. Strongarm is free for personal use.|
|Talos Aspis||Project Aspis is a closed collaboration between Talos and hosting providers to identify and deter major threat actors. Talos shares its expertise, resources, and capabilities including network and system forensics, reverse engineering, and threat intelligence at no cost to the provider.|
|Threatglass||An online tool for sharing, browsing and analyzing web-based malware. Threatglass allows users to graphically browse website infections by viewing screenshots of the stages of infection, as well as by analyzing network characteristics such as host relationships and packet captures.|
|ThreatMiner||ThreatMiner has been created to free analysts from data collection and to provide them a portal on which they can carry out their tasks, from reading reports to pivoting and data enrichment. The emphasis of ThreatMiner isn't just about indicators of compromise (IoC) but also to provide analysts with contextual information related to the IoC they are looking at.|
|VirusShare||VirusShare.com is a repository of malware samples to provide security researchers, incident responders, forensic analysts, and the morbidly curious access to samples of malicious code. Access to the site is granted via invitation only.|
|Yara-Rules||An open source repository with different Yara signatures that are compiled, classified and kept as up to date as possible.|
|ZeuS Tracker||The ZeuS Tracker by abuse.ch tracks ZeuS Command & Control servers (hosts) around the world and provides you a domain- and a IP-blocklist.|
It is hard for me to track on everything in the API space as a one man show, but one thing I keep an eye on, but rarely add to my research (yet) are the increasing number of job postings I come across that have an API focus. An interesting one this week, which follows a new vein in the API space that I'm mining, is out of the applied science technology research organization Battelle.
They have an interesting job posting for a cyber computer scientist with Battelle’s Cyber Innovation Unit (CIU) providing "a broad range of products and services including Cyber Intelligence and Counter Intelligence, Cyber Research and Development and Fundamental Science, Onsite Advisory Services, and Mission Focused tools including Embedded Systems Reverse Engineering."
What brought the job posting up on my radar is one of the responsibilities being for "data APIs", and a requirement (mentioned twice) for having API design skills. The role APIs are playing in the areas of cyber security, cyber intelligence, and apparently "cyber innovation" is why I recently added cybersecurity as a research area to my API monitoring stack. It's not that APIs are being hacked, it is that they are being used for research, analysis, collaboration, communication, and other layers of the cybersecurity theater.
If I keep coming across interesting job postings like this I may have to start tracking on API jobs more, as the overall volume of API jobs that are coming across my radar is increasing, but also the interestingness is also trending upward significantly.
I am slowly getting back into the routine of doing my weekly roundups. It has been a while since I published any, even though I regularly do the work. While I was going through this week's roundup of items I curated, I thought some of the blockchain related goings on were particularly interesting.
- IBM Launches Cloud Services for Blockchain Deployments (www.infoq.com)
- Bitcoin users have been warned about a potential attack by government hackers (www.businessinsider.com)
- Hackers Say They Hacked NSA-Linked Group, Want 1 Million Bitcoins to Share More (motherboard.vice.com)
- Japanese concerns over blockchain technology skills shortage (disruptiveviews.com)
- Blockchain and bitcoin companies raised $290 million in the last 6 months(www.businessinsider.com)
- New Crypto-Currency Rewards Users For Participating In DDoS Attacks (www.itsecurityguru.org)
Not sure about you but I can't help but think that has the makings of a pretty interesting economic engine for the cybersecurity industry. You have government hackers, organized hackers, rando hackers, concerns around having enough talent, investors pouring money into the space, and 1000lb pound gorillas making firing up their digital factories.
I'm guessing that blockchain and cybersecurity are going to go hand in hand, and be a very lucrative endeavor for a select few.
If you think there is a link I should have listed here feel free to tweet it at me, or submit as a Github issue. Even though I do this full time, I'm still a one person show, and I miss quite a bit, and depend on my network to help me know what is going on.