DSL not for VOIP

I recently (temporarily) switched from high-speed cable to DSL. The local cable companies do not service my present address. I've been using VOIP (Voice-over-IP) phones for the longest time without issue. I recently upgraded my phone system and on ADSL the voice quality is horrible.

I suspect the reason for this is that the local phone company doesn't want you to use VOIP because if you do, then you're not paying for their local phone service. Additionally, they're likely trying to filter any attempts at people streaming any content from their local networks. To combat this, by default in most of the DSL modems the phone companies, by default, have configured network communications to work with UBR (Unspecified bit rate), which means that the line variably bursts with a *potential* high-speed of your maximum internet speed.

For Voice-Over-IP communications the setting would need to be CBR (Constant bit rate) since you're streaming audio over the internet. So for anyone considering using Vonage or Magic Jack on a DSL system, you're probably better off paying your local monopoly to have a POTS (Plain Old Telephone System)... until of course they upgrade your circuit to fiber.

Massive fail in terms of User Experience and User Interface on Tivo Premiere 4.

As subscribers to Tivo's service we recently "upgraded" our 4 year old Tivo HD DVR to a Series 4 Premiere. Upgraded is their phrase not mine. Having done computer upgrades for years and even cross platform upgrades between Windows and Macs “upgrade" is not really the word you're looking for. More appropriately, buying a new Tivo and transferring existing service to it is a great way to waste a couple of days of your weekend.

A rough transfer to a Tivo Premiere:

1. If you have shows recorded on your old DVR and "upgrade,” you have to maintain service on the old DVR in order to access the shows on the old DVR over your local network.
2. If you attempt to maintain service for the month so you can transfer recordings to your new DVR expect to sign-up for a contract. Tivo will not allow you to extend service as a gesture of good will. In fact Tivo’s sales and support staff laugh when you mention to them that you would like to maintain both DVRs on the account for 6 hours.
3. My new Blu-Ray player has a USB port and I can watch movies on it. My Tivo Series 4 has a USB port and they've disabled any functionality outside of extending user input interfaces... meaning if you copy all of your old shows (in Tivo's proprietary format) to an external hard drive via Tivo Desktop on a computer, when you plug the drive into another unit you can not access those shows.
4. Their new user interface is abysmal... if they didn't provide you with the option to use the old interface you would likely throw your Tivo into an abyss.

When the new Tivo arrived I had already researched the massive failure on Tivo's part (I've been a long time user of Tivo so I've come to expect failure). I had a few major hurdles to overcome.

1. I had to transfer my cable card from my existing Tivo to the new Tivo. Since it's locked by the cable company to prevent theft or copying I had to contact them and walk through the painful transfer while I tried to explain what we were doing.
2. I unplugged my old Tivo from the ethernet connection so it would not call the mothership to find out it was no longer loved. I thought I had it fooled. When I removed the cable card the machine went into a panicked state and did everything short of mandating that I contact the Tivo service to reset the Tivo. Luckily I restarted the Tivo by unplugging it and the behavior went away. It of course was unable to watch live TV because of the lack of a cable card, but I could at least get to the main navigation again and watch the already recorded shows.
3. Since I had unplugged the old Tivo I was able to plug it into a local network (not connected to the internet) and access it with Tivo Desktop. I was after several hours of experimenting able to finally get the Tivo Desktop Service to successfully copy all of my recordings from the old Tivo.
4. After copying my old recordings from the old Tivo I figured I would show them on the New Tivo by streaming them over the local network. For about 20 seconds my workstation showed in the "Now Playing" list on the new Tivo. I could browse the list of videos from the old Tivo. When I selected a video the Tivo said "This video is no longer available." Then the server disappeared from the New Tivo's interface. I have not successfully connected the Tivo Series 4 Premiere to the Tivo Desktop Plus application since.
5. Online during my research I noticed on the Tivo website there was a feature to transfer the Wishlists and the Season Passes. This data is not “saved” on their website, it’s simply read from an activated unit. At the point a unit becomes deactivated, all data on the Tivo website is removed.

Since I’m a member of the Tivo Advisors committee, they never really give us a chance to indicate what we want from their service, they’re only interested in what type of car I want to buy, or what type of movie I might be going out to see. So here are my recommendations to make the user experience for Tivo customers much better.

1. When you offer users an “account” that they can use online to schedule their programs, keep a copy of this data. Since it pertains to your users, this account info should NOT disappear or be tied directly to a unit.
2. Users should be able to setup profiles. Most users expect that they can record things. If a family of three has three different people, they likely have 3 different interests. It’s in Tivo’s best interest to maintain information on these 3 interests so they can use the demographics for their marketing practices.
3. Stop locking your machines. It’s running a crippled version of Linux. Everyone knows this. Let people use the devices. You’re much more likely to have happy customers if they can actually use the machines they’re purchasing rather than having them set like bricks when a new Tivo comes out. No, I will not punish someone else I know and give them my old Tivo.
4. Learn about who your users are. If most of your users are using Apple products, then make your units work like Apple products do. Make it easy for them to download updates at their wish. Make it easy for them to connect to the machines on their local network to get stream their local content.
5. Stop putting ads in the main list. The whole reason I have a Tivo in the first place is so I can filter out ads. The last thing I want is MORE ADS.
6. The Tivo recommendations are bad. Take a hint from Netflix. Let people provide you with their likes, then you can suggest shows that way based on shows that other people like similarly. Don’t recommend something to someone because a company pays you to. This comes across as less helpful, and more advertising.

Web Form Security: Reasons behind online attacks

Why am I being hacked?

To really know what you're dealing with you have to get inside the head of a script kiddie or a hacker if you want to actually "secure" your systems. Since there are so many factors, many of which that are usually out of the control of most individuals, I'm using the phrase "secure" loosely. From a web or online security standpoint I've worked with several companies over the years, usually in a post-attack analysis, trying to determine what happened, how to recover (if possible) and how to harden against the attack again. Companies often do not spend money on security before an attack and say things like "It's never happened before." or "Why would they target us?" or "No we haven't been hacked." when in actuality they have.

There are several reasons why someone or a group might want to take over a webpage, a blog, a webserver, or a MySQL database server. Here are a few of the reasons I've experienced myself for why someone would exploit a site or page.

Web Real Estate

Mission critical systems that rely on a database need to be secured. Not only is there the risk of someone data mining a database of personal data, but there are also risks for the database server that contains the database and/or the website servers that host the site receiving or displaying the data. One of the ways people can cause havoc on a server is by using an SQL Injection Attack. In November of last year I wrote a post about SQL Injection Attack Precautions. It talks about who's ultimately responsible in terms of securing a system since usually in most cases the blame for an attack is spread across several people.

How could web real estate be at risk? If someone looks at a form for a search, they can assume that it is connected to some sort of database. Blindly hacking at the form, they will not be able to tell if the database is a PHP array, an SQL database, or an XML file until they receive an [un]intended response. Through passing unexpected characters into the form they can potentially break the form, cause a stack overflow on the server (effectively crashing it), or break the application that is handling the form. Something like putting a server into an endless loop can bring a server to its digital knees. This usually involves  passing escape characters to add extra slashes, closing quotes (single and double), programming language terminations, or by passing HTML code into the form. Passing empty form fields can break some forms, while others can be broken by simply disabling Javascript.

When a web form is broken it returns valuable information to an attacker about the structure of the system, the type of server services running, and the quality of the code on the system itself. In my experience most websites with easily hackable code are frequented more heavily by would-be attackers and script kiddies than sites that return no errors or information to an attacker. Since most modern web servers are hosted in server farms with high bandwidth connections, to outside attackers it will more than likely be the same payoff for hacking a sophisticated site versus a simple site. They both offer the similar  bandwidth and server resources and they are usually designed to be managed remotely so there is little chance the Administrator will spot the attack. If an attacker sees an increased level of security, they're less likely to attack a server simply because their efforts will be undone much more quickly or they'll be caught because they will have to try harder.

Web "Street Cred"

Just like the real world, online hackers need notoriety. That being said, there are individuals in the hacking community who love a challenge. Some websites such as tech blogs, newspapers, social media accounts, video streaming websites and social networks are going to be more at risk for someone trying to replace content or services simply to make a name for themselves. There are far more people looking to become famous from a hacking attempt than there are people looking to steal information and sell it on some black market. The skill sets required for guessing a password to take over a page vs. actually deriving unencrypted usable data that can be sold are night and day different. There are quite a few apps in the open that will crack or guess a password. There aren't very many individuals that can successfully write a root kit. Sometimes an attacker can simply guess the password to get in and look at the code. The guys who do it for a living will not be bragging about it unless they're making a sales pitch for paying work behind closed doors. You will see script kiddies doing it so they can make a name for themselves (think Anonymous).

Political Reasons

Some "groups" like Anonymous take pride in bringing down sites and exploiting pages and accounts with opposing views or showing companies and corporate conglomerates that they have glaringly open holes in their security. Search for "Anonymous Hacks Burger King Twitter" on Google. While there likely are real hackers that operate under the "Anonymous" moniker, most of the exploits I've seen are pretty amateurish. If Anonymous were really a serious group there would more than likely be now more online trading (or stock market for that matter).

Bad SEO

Some people just want more links for their own sites. These people can be spammers and sometimes they're legitimate businesses that have paid for a service that they themselves weren't quite sure on. In the past there was a practice of spamdexing where a website listed in major directories or topics pertaining to the contents of the site would be picked up and rewarded by the search engines. Fake sites and phishing sites soon caught onto this. The search engines changed their policies, but sometimes in countries throughout the world word doesn't travel so fast through translation. Many "SEO specialists" mention that they can get a site listed through link sharing. This is more than likely how if they are overseas.

An example of spam-dexing from the Search Engine Journal (3/12/2013)

"There are many sites with spam on their sites that can’t see the links that they are showing where you couldn’t see unless you went into the code.  Google bot shows that a Top 50 University has “cheap viagra pills” on their main page."

To find out which one you can search for University Viagra on Google.

Data Capturing including Credit Cards and Social Security Numbers

Some people are a little more secretive about their exploits and they will hide code on a system to take advantage of web visitors and traffic. This may take the form of database copying or replication (if the site is storing e-mail addresses, credit card numbers, or sensitive data). The attackers may send copies of the real submissions to their own server. They may monitor statistics from the site (for a competitor). Some attackers inject malware into the code so they can infect user computers. In a previous post I talk about the hacking of clothing manufacturer Calvin Klein and how I started receiving SPAM from the newly created e-mail address I used for them the day I signed up. Calvin Klein of course denied any knowledge of this or interest in rectifying the issue.

Additionally when someone is actually capturing all information to a system on the system itself, any information passed is vulnerable. This includes Social Security Numbers, Credit Card numbers, and anything else that may be submitted (student ID numbers). Depending on the type of site, this is a huge risk to clients, customers, and worse... the brand in terms of PR backlash.

Bot Net  

Web servers can be powerful, plentiful machines just ripe for harvesting. Located on massive connections there is very little that can be done to track multiple machines requesting orders from the controlling system (the requests can look like normal web traffic in a packet filter). In numbers, compromised machines can become a powerful collective. Why not run an application in the background on someone else's web server to make it control countless drones while it goes on serving a webpage? This does actually happen. Usually the attacker will install something called a "root kit" which is an app or framework that is undetectable that runs in the background. This allows them to control the server and exploit the bandwidth and resources available to the server. The web page may be up and running and unchanged, so the owner usually won't find out until there is a knock at the door because the machine was used to exploit someone else's, it was controlling countless other machines or worse the website goes down because the ISP pulled the plug at the request of a government or after their own inspection and determination of high traffic. Once a root kit is installed it is easier to use a new machine than it is to clean off the root kit. Without examination the exploit the attacker used may still be in place. It would only be a matter of time before the attack exploited the machine again.

So what are the real risks?

Most of the time the attacks come down to bad password management policies, or use of an unsafe network by someone to log into a website control panel or administration panel (think Starbucks). Every once in a while someone is hit with an XSS attack or a/an [My]SQL injection attack, but this requires someone actually trying to hack the server. Passwords can be captured in open places like airports, coffee shops, hotels, vacation resorts, cruise ships, and on any other unsecured WiFi networks with free applications on the web. Be smart and use strong passwords longer than 10 characters in safe / secure locations and more than likely there will be no issues.

Web Form Security: Stopping Spam

Sites are often hacked through poor
web form implementation.

This is the first of a multipart series on securing web forms. One of the best ways to approach web form security is thinking about the form through the eyes (or the mind) of the attacker. What do they think they can gain from the form? In this first posting I'll talk about SPAM and why it happens and potentially how to stop it.

SPAM (non-solicited e-mail) has been a problem almost since the beginning of the Internet. It costs companies billions of dollars in wasted bandwidth and resources such as anti-spam firewalls, hardware spam filters, anti-spam e-mail filtering services, and lost employee time. Since 1995 [the] HTML [language] has allowed for an input tag and web forms for uploading images, files, and supplying different types of input. As web bots or crawlers became more prolific around the beginning of the century companies and private individuals began turning to HTML [web] forms to reduce or cut the amount of SPAM (unsolicited e-mail) they were receiving from web-posted e-mails (e-mail addresses that were actually visible to a browsing visitor).

An e-mail address in text on a website is surely to be added to hundreds if not thousands of spam e-mail queues.

Initially having a form that would post to your e-mail account was enough to stop a lot of the spam, but as data-mining became much more invasive, form elements containing email addresses were being mined specifically for those addresses and more spam continued (e-mail addresses in web forms are available as text to anything that can parse HTML). Then there were exploits of people injecting information into web forms and relaying messages through Perl server-side CGI scripts. Once exploited, someone could easily send e-mail from a webserver and have it look like it actually came from the company hosting the form. Most modern web forms are processed by a server-side component [language] such as PHP, ASP, ASP .Net, Ruby, Python, and so forth. Even though with server-side processing it is much easier to filter the information coming in through a form, many times a "spammer" can beat a form by simply completing the form as a person would.

E-mail Addresses
Some web forms still use outdated non-industry standard code to submit a message from a website to an e-mail address using the "mailto" option. These forms will typically open a default e-mail client (such as Outlook or Mac Mail) upon the user selecting the submit button. The reason this needs to be avoided:

1. The e-mail address is visible to anything on the internet.
2. Some people use webmail and this opens a program that may confuse or irritate them.
3. This e-mail address can be added to a SPAM list or be set as the recipient of e-mail bounce backs from SPAM or spoofed e-mails.
4. E-mail gathered from websites is sometimes sold in online mailing lists to people who believe they are receiving a list of targeted e-mails when in actuality, they are using mined lists.

Steps to reduce spam from the web

1. Remove all text-readable e-mail addresses from your website. To check this, open a webpage in a browser and "view source." If you search for the @ symbol in the code and find it, make sure it is not in an e-mail address. If you're responsible for programming the site, replace this option with something else. If you're not responsible for maintaining the code, contact the programmer and ask them to go about creating a form for e-mails from the website. If someone needs to contact you they will find a way. If you are a business, do not rely solely on your [web form] e-mail as your main point of contact.
2. Provide a working form that can filter your messages from the site. Most [web] hosting plans come with a server-side language component that can be used to filter the messages. This language may be PHP, ASP, ASP .Net, or Perl. Servers installed by companies internally where a website is hosted locally also come with these options already available by default.
3. Search the internet for company listings that contain your e-mail address. Sometimes this may include corporate directories, trade publications, web domain registries, and message boards. You can ask them to remove the e-mail address and replace it with a link to the web form.

Reducing other spam

Many of my clients publish their e-mail addresses in print on a variety of materials. These e-mail addresses more often than not go to some mass distribution group in their e-mail server. When a spammer sends a spam message to this e-mail account it is routed to more than one person. For every person the message is sent to there is a copy in their inbox (or Spam folder) on the mail server, not to mention potentially in a sent box from the distribution group, or in the inbox of the distribution group if it is setup as an individual that forwards rather than a forward-only box. If the person forwards to their phone and doesn't use a connector like IMAP, then there may be multiple copies of the e-mail per person as well. All of these messages [usually] take a tiny amount of room, but in greater numbers they can take a lot of space on a server or a local workstation (IT Real estate).

1. Try to limit the e-mail recipients for addresses in print to only the people who maintain the list. Obviously business cards will need to have an e-mail address, so I'm talking about brochures, flyers, forms, letterheads, envelopes, and advertisements.
2. Make a group-accessible mailbox for any inbound e-mails rather than distributing them through the mail system. This way any person in the group can delete the e-mails from the single location. Back this up in the event of accidental deletion.
3. When printed documents are available online in PDF form they can be mined for e-mail addresses just as easily as a web page.
4. Follow-up e-mails to submissions should come from a no-reply box or something that can be checked for mail submission, but not a distribution group.

Stopping in-bound spam with a web form

When securing a web form, there are a few things to consider.

1. The person completing the form may not be a person at all. It is possible to submit information to a webserver via the POST and GET methods without using a web browser.
2. Where is the submission of the form ultimately going? CRM, e-mail only, a database?
3. If a [real] person can't complete your form because it is too complicated there is not point in actually having the form. They won't use it or worse, they'll go somewhere else.
4. If your form relies heavily on a client-side filtering (eg. Javascript), that scripting language can be [more than likely] disabled. If it is disabled the filtering may no longer happen. Can they still complete the web form?
5. Non-filtered web forms are [some of] the biggest risks to databases and corporate infrastructures. 

There are a variety of things that can be done with PHP, JSP, ASP, and ASP .Net (commonly found on Microsoft Web Servers) that can dramatically reduce the amount of Spam you receive from a form.  

K.I.S.S.

I was unfamiliar with this phrase, but one of my clients said "KISS... Keep It Simple Stupid," in response to their bad web application I've been repairing (previous vendor). I always try to setup a form to be simple to use for the end user and the recipient of the form details. Bad web forms [overall] cost companies millions [maybe billions] every year. If people can't complete a form, then sales can be lost, searches can't be made, potential customers feel the bad service is already starting and they are not even a customer yet.

Hack it.

I test the forms heavily to make sure they work. I try to type in incorrect information: I misspell things, omit fields, forget to put the @ symbol in e-mail addresses, and fill the forms with data in the wrong fields. Usually I weigh the feedback from the form to see if they are purposely entering misinformation or if they enter it humanly impossible. Then I check to see if the form was actually submitted. If it wasn't, then why?

Autofill

I use the auto-fill features to complete the form and submit. I see if the auto-fill features of my browser actually complete the form. Most people do not have a lot of time to fill out forms, so if you can make it present them with standard fields they're accustomed with they can complete the process more quickly. Auto-fill works by using some normal field names to gather information, then when those field names are used again, the auto-fill component of the browser(if enabled) will present the user with their past responses.

Don't reinvent the wheel

It's a web form. People are used to completing things in a certain way. Present the information in an intuitive method for the audience. If you have clients from all over the world, don't mandate a state name, or a county. If they're only supposed to be from one country, then you can omit the country field. Use words that translate into other languages easily. To see some of the field names for common web forms check out sites that people will use on a regular basis. Examples include sign-up forms on sites like UPS, postal services, Facebook, Twitter, and LinkedIn. Use your browser to "View Source" on the forms for those pages and see what the fields are named. If you name your field "client-email" it will probably not auto-fill, but if you name it "email" or use the HTML5 field type of "email" then it should work without issue when it comes to using auto-complete or auto-fill.

Avoid client-side language filtering

Because they can be easily disabled I recommend avoiding languages like Javascript in the web form. I've seen forms that have interactive elements that tell you whether the different components of the form pass a test before they can be submitted. The drawbacks are that not everyone has Javascript enabled, sometimes these things become annoying by removing elements from my message and telling me I've completed something inaccurately before I'm done with my submission. Also browsers implement Javascript components in different ways. For instance a company with a policy of using older versions of Internet Explorer on their workstations rely on ActiveX controls for AJAX( the scripting interface for dynamically checking a field without someone submitting the form with Javascript). These non-signed ActiveX scripting components are disabled by default for security reasons. The people who would use your form to submit their message may not be able to complete the form if it relies on Javascript or AJAX.

If you do decide to use AJAX, remember that the handler for the AJAX is available in the code. Anyone who wants to take over your server may use this as their point of attack. By submitting to the handler directly and bypassing the form altogether they can potentially find weaknesses in your application, server, or code rather quickly if they're using a bot net (group of compromised machines).

With that being said, avoid Flash as a web form. It uses a client-side scripting language based on ECMAScript (the basis for Javascript), not everyone has it installed, it doesn't work on all mobile devices. Flash is buggy at best, and Flash can easily crash a browser on its own. There is no reason to use Flash as a web form. Also there are ways to beat whatever filtering the Flash application is doing prior to sending to the server, meaning Flash might actually the Achilles' Heel of the safety of your server. Just as someone can see what the AJAX web component does without using their browser, someone can download a Flash decompiler, use a header checker in a browser like Firefox on a Flash web form, use a web debugger like FireBug to watch information transferred, or open packet filter like WireShark to see what information is being transferred to the server (if you're using a stand-alone Flash application on a DVD).

Serverside language filtering

When you're filtering server side, be careful what information you give back to a potential attacker. Do not allow them to enter code into the form and the give it to them as an attempt to have them correct it. Also it's good practice not to force someone to review the contents of their message.

PHP (one of my favorites) comes with the various functions to take advantage of Regular Expressions (another language for searching and filtering). With regular expressions or RegEx, some new programmers who are given the task of hardening a web form may see this as a viable option for screening the fields massively. This isn't a bad mentality, but it does depend on what you do with a failed response. Bounce someone for an incorrect field entry and you may lose a client or potential customer. If for instance the user enters the information in their own language (eg. Chinese) and the programmers assumes their own language (eg. English) for filtering requiring English characters, then the potential user may become a false-positive as an exploit attempt.

When I check fields I try to make it something a little more obvious. Here are a few things I check for:

1. Do "name" field submissions contain numbers? (not typically, even for Edward II they use Roman Numerals)
2. Do "e-mail" fields have "@" symbols and at least one period after the symbol? (a necessity)
3. Do phone numbers only contain numbers? (What about +, - , Ext, Extension, x, '.', They might contain any of those) 
4. Does the address contain a space character? (A necessity)
5. Is an address really required? (If it's not, don't mark it as such, and don't force someone back to complete it.)
6. If something is not required, but entered, does it still conform. (eg. Address isn't required, but they filled it with random garbage... they're probably a spammer.)
7. Do the comments contain URLs? (Maybe a spammer? They might be telling you about a problem on your website.)
8. Am I expecting BBCode(something usually found in web forums) in the comments? (Probably not, this is more than likely a spammer)
9. Did they put a space in their name? If they didn't, is it okay to accept information on a first name only basis?
10. Did they include anything that is obviously an SQL injection attempt? (eg. "; delete from users where 1=1")

Are they real?

This is a huge question in terms of securing a web form. If the attacker is using a program to auto-complete the form to submit Spam, or if they're using a group of computers (bot net) to attack the form and bring down the server, then how can you stop it? Simple. See if they are real.

Typical Captcha

CAPTCHA is a bad thing in many ways: It makes people angry. It's hard to complete. It's not always readable. It's a complete waste of time. Captcha stands for "Completely Automated Public Turing test to tell Computers and Humans Apart." It's basically a quick fix for trying to guess whether someone is human. International users? Avoid Captcha.

Hashing and tests for human skills

Just as CAPTCHA makes an attempt to make people prove they're human, there are a couple of things you can do behind the scenes to see if someone is a real person.

1. Did they completely the form in a humanly possible time?
   Depending on the required information from the form, and the type of information expected, run a few timed tests. Use things like auto-fill and auto-complete to try and beat your human times. In my experience, most bots will on first attempt complete the form in less than 2 seconds.
2. Did they complete the form using two different IP addresses?
   Simple... check their ip address and, pass their IP to the handler page. Check it on the next page for a change. Oh, but what if they modify this IP address you're using.
3. Did they even use the form? Check this with simple hashing.

Most programming languages include a function called hashing. I use hashing as a test to see if someone is altering my expected information or if they're modifying anything that I'm setting myself. If they are, then chances are they're not using my web form unless they're using a browser plugin that lets them rewrite the HTML components.

Some of the tricks I do on the form:

1. Pull their IP address. Include this in the info headed to the form. (If their location changes they're using a proxy or they're not using the form.)
2. Pull the timestamp for the viewing of the page. Include this in the info sent to the handler. (If it's too short, then they're not real. If it's too long, then they're not real.)
3. Pull the browser's User Agent. Include this in the info sent to the handler. (Does it look like a real browser? Does it say cURL?)
4. Pull a random number and hash it also. (This will not be used at all.)
5. Don't label these things in an intuitive way, but rather place comments in the serverside code that indicate what you're expecting.

Hash these three things together with something only known to you (a special word or phrase) and submit them in a different method than the rest of the form, meaning if you're using POST variables for submitting the contents of the form, then submit the Hash with the GET method variables.

Some tricks on the handler:

1. Pull their IP again. Does it match what was submitted?
2. Pull the new timestamp. Minus the old timestamp and see how long it took.
3. Check the user agent. Is it a real browser? Does it have a keyword in it like Bot or cURL? If so, then it's more than likely not a real person.
4. Do the new hash with the info passed from the original form. If it matches, then you know the form information wasn't altered. If it doesn't match, stop processing the form.

What happens if I don't hash my submissions?

I did this myself when I was first trying to beat the spammers. I started getting spam emails that were submitted 3 years prior or 100 years prior or 2 years into the future. Investigating, I noticed (in my custom statistics app) that spammers were reloading the form over and over again (likely in view source mode). They would alter the values of the timestamp and resubmit the form. Then they would do it over and over again filling my inbox. When I viewed the source and did this myself I noticed that they were watching the hidden values in the forms. They were seeing if they changed and if they were timestamps or whatever. If something didn't change it was a straight hash of something provided before the form was submitted (IP Address or User Agent). If it did change then it was a timestamp or a random number. I tested by hashing timestamps initially. This led to the same results. The spammers were guessing my hashing method and hashing the timestamps and presenting them to me, edited, in mass. I started hashing random numbers as a salt (cryptography meaning) with MD5 and I noticed that the spammers stopped filling out the form for a while. They couldn't figure it out, so rather than trying they would go elsewhere. Some still tried. Ultimately the only way to beat it was to complete the form as a human being. Some still do.

About 98% of my web form spam stopped when I started hashing my results and testing thoroughly. I capture the failed attempts in a text file (for logging, false positives, and form hardening stats) and pull the country from their submissions based on IP address. Most are from India, China, and Russia,with a few from the middle-east.

The whole picture

The best method I've found for beating spam with some of my corporate clients is the hashing method I've described, and I use a scoring system to see how bad a potential spammer might be (through filtering). If they don't complete 2 or 3 of the form fields correctly then they get a likely spam score. Certain things are a dead give-away... no @ symbol and they're a likely spammer. "Viagra" in all incarnations... (\/iagra,viaGra,v!agra, etc). You have to be careful if you're blocking words. "Cialis" is in the word "speCialist." I include the results of my spam scoring in my text-only files and copies sent to my clients. We occasionally see a false positive in a foreign language, but for the most part the Spam scoring is dead on.

In the next segment I talk about Why people attack sites online.

Check back for more posts. I'll update this entry when I add more to the series.

Until later,

-Chris

Technology's advancement requires competition.

I'm 100% in favor of a company starting out small with an innovative idea and then expanding. If this happens in a way so that little competition happens for the innovative company it's something magical to behold. Everyone benefits from competition however.

Without competition a company can set prices to whatever they want. If an innovative company has a new product and no other companies step up to compete, it could be because other companies lack funding, the knowledge, or they may see no benefit in competing at all (for a product they do not believe will be successful).

More and more in this day and age companies aren't getting ahead because they have more superior products or because the people buying the products have a heightened sense of brand awareness, but rather because a company benefits from information or services illegally or they pay other companies not to use the competition.

When a person knowingly supports a company that has ill-gotten gains, this helps and endorses the company to keep doing what they were doing. By having an unfair advantage a company can put all competition out of business and then set prices to control a market. If the item in question is technology, they can control all prices globally. Also without competition, technological advancement is in the hands of the only company left standing. If the company decides not to advance because it's not in the best financial interest of shareholders, then the results could be detrimental to a product line, a piece of technology, even society itself (just think if one company controlled the water supply... see Wikipedia for Water Privatization).

AMD Advanced Micro Devices and why you should not buy Nvidia or Intel (at the moment)
Many companies benefit when they hire a disgruntled employee from a competitor. They receive inside knowledge of the inner workings of the competition. They also benefit from any project the employee might have had knowledge about, not to a degree that the company can copy the technology entirely or beat the original company to a patent (unethical), but they can prepare for the competing technology, software, or product to be on the market and find ways to innovate and compete ethically and legally. This is the reason that companies have employees hire a non-compete and also clauses that state that anything you work on during employment with the company is the property of the company done as "work for hire." I myself feel non-compete clauses should be illegal themselves, but in most cases a company will be hard pressed to keep an ex-employee from obtaining gainful employment in their field of expertise. Work for hire is something that should be allowed if the company is funding the research, but if the company can show no receipts for the time the employee came up with the idea, then it should belong to the employee.

Sometimes however companies don't receive information legally, but instead pay recruiters to tempt employees of the competition into selling inside information before the employees have left the company. Insider theft and espionage not only cost companies billions, it can put a company out of business and even hurt everyone involved.

AMD Advanced Micro Devices stock values 1/28/2013.

Two cases have come to light in the past years involving AMD and unfair practices against their business. In a lawsuit filed 1/14/2013 - AMD vs. Feldstein, Desai, Kociuk, and Hagen - AMD is seeking damages and injunctions against the 4 people involved who allegedly sold inside information and collected data from the AMD database to AMD's main competitor in the graphics card market, NVidia. If only one person had sold the information to NVidia, or attempted to sell the information, then there might be the case that NVidia had nothing to do with the case and the person selling the information might have been opportunistic. Since four people sold information, it looks more like NVidia might be paying these people (and recruiting) information for ill-gotten gains.

The second case that comes to mind is an Antitrust issue between Intel and AMD. There was a "complaint" filed for NY vs. Intel where they go into detail about Intel suggesting to their clients that they stop using AMD chips. In the EU there was an Antitrust case filed against Intel in 2009 where the courts ruled in favor of payment to AMD. Intel's counter "Intel takes strong exception to this decision. We believe the decision is wrong and ignores the reality of a highly competitive microprocessor marketplace..."

In short, No, it is not innovative to pay off the market and keep companies from purchasing from your competitors products.

Is it okay to buy anything Apple branded?
While I definitely like the road Apple has taken with their machines recently in terms of speed, I give second, third, and even tenth thoughts to buying Apple products. Apple has become a company that ignores human rights when it comes to building their portable devices. Another reason is that Apple exclusively uses Intel chips in their machines and do not allow installation of their operating system on any other platform (including AMD). From Apple's EULA for Snow Leopard:

"You agree not to install, use or run the Apple Software on any non-Apple-branded computer, or to enable others to do so."

When companies (Psystar and PearC) selling hardware of their own branding with the Mac OS operating system installed were sued by Apple, the supreme court found that the use of the Apple Operating System on Non-Apple hardware was a violation of the DCMA. Meaning it's illegal. This makes me wonder if the Librarian at the Library of Congress has received any compensation for helping Apple to become a monopoly in this regard since the Library of Congress controls the DCMA (Digital Copyright Millenium Act). Because of this I have only purchased low-end Macs for checking email, but maintain an AMD 12-core server as my primary workstation.

Hybrid Postal Delivery Services: How they destroy brands

I recently ordered an upgrade to one of my workstations from a “local” vendor. They’re about 60 miles from my present location, just outside of Chicago. Most packages in the greater Chicago area being sent through the United States Postal Service take a maximum of 3 days from the time they’re sent, in my experience. This usually involves going from a local post office, to a main sorting facility, back to the destination post office, and into the hands of the postal carrier. Three days is on the high-end, as it is usually only takes two. This all depends on whether the address is handwritten or if the sender printed a barcode with all of the CASS-Certified presort information detail on the label. (Hand reading and sorting adds time to delivery.)

When I was making my purchase from the website, (I'm giving them a second chance hence the failure to mention them directly), I was presented with a couple of options: FedEx 2-day which would cost me an additional $15, FedEx Overnight Air $30 (no air involved for a local delivery), and several other highly expensive services. I trust the Postal Service very little, but rather than paying for extra non-essential services when my package could be delivered in two days using the normal postal system, I elected to use the “free” service which guaranteed 2-3 days.

When I received my receipt the vendor indicated the 2-3 day delivery and two and now three days have come and gone. My dilemma is that the people I ordered my package from, rather than using the standard United States Postal Service in a local, traceable method, decided to use one of the new hybrid services, in this case UPS SurePost 'Saver.' I HATE seeing this as the free option for local shipping because it almost definitely means that the package is going to be lost and take an extra few days. FedEx has a similar service call FedEx SmartPost… equally as bad (if not worse). When I use either of these services I end up seeing my package within 2-3 miles of the office for 2-3 days before it is finally delivered. Something about the process makes the postal service or the shipping service delay the final delivery.

After looking at the tracking detail last night and expecting my package to arrive today, I went down and met my postal carrier at the box and surprise, surprise... no package. He looks at me rather puzzled. I look at him rather puzzled and bid him a good day. He’s a nice guy, so are my local UPS drivers... it's not their faults... it's the logistics.

Upon returning to the office I go in to check the tracking detail. Apparently my package was “ROUTED TO WRONG LOCAL POST OFFICE. PACKAGE WILL BE TRANSFERRED TO CORRECT POST OFFICE FOR DELIVERY,” according to the UPS website. When I called UPS, rather eager to pick-up my package in person (because I’m tired of waiting), the person on the phone told me that my package would be delivered either today or tomorrow and that they were on top of it. When I asked if I could pick up the package, they said they weren’t sure where the package was exactly... a breakdown in the tracking detail between both services involved, in this case UPS and USPS.

So this brings to light several reasons why these services DO NOT NEED TO EXIST AT ALL. There are no savings using this model for anyone: shipper, receiver, or the shipping service(s). When a company loses a package or misdelivers a package due to the complexity of the shipping logistics it has the potential to smear all of the brands involved. That costs companies money (think Billions). In fact, here I am smearing their brands, DO NOT USE UPS SurePost or FedEx SmartPost ‘Saver’ Services for delivering packages to your customers or clients. They will find other vendors. Offer simple, yet-traceable delivery services. I may not purchase anything else from the original company for fear of not receiving it on time (or at all). I will avoid the UPS SurePost ‘Saver’ delivery service, like the plague, and try to find another vendor that will simply send my package to me, timely without added expense and patience required on my part.

If one were to go in and read the countless reviews on Amazon.com, Newegg.com, ebay.com or several other websites where reviews abound, they will notice a pattern of people who give an item a low rating simply because of a shipping delay. This not only hurts the success of the product (manufacturer's brand) that they are berating, but also the reputation of the company (seller's brand) that is selling the product. This is no doubt because the people doing the ratings have no concept of what they are doing, nevertheless it happens and is also costly.

When a package that should normally only touch two local post offices and a main sorting facility, bounces through three UPS sorting facilities, a local UPS branch, and two local United States Post Offices, and multiple mail carriers there is an increased risk of the package being mishandled, misdelivered, lost, stolen, and/or destroyed.

My recommendation if you’re UPS, USPS, FedEx, or Any Company that wants to have customers that spread good words of mouth about your products and services, then DO NOT use any of the hybrid sending services (or provide them) because unlike the normal services customers have come to love and expect, these complexities to the rather simple purchase and delivery model are a risk to all.


That's all for now.
-Chris

Information Sharing - A Double-Edged Sword

This posting was sparked by a few new exploits on the rise, a Java exploit and a couple of Ruby-on-Rails vulnerabilities. I found out about both of them from Dark Reading.

I've been using computers for a very long time (31 years) by technology standards. One thing I've become accustomed to is regular updates and patches to systems, programs, and apps. Sometimes the systems that need to be patched aren't the systems that people themselves might have access to, but they may be a web server, a mail server, a programming interface, or even a server-side plugin.


The good
The reason these things need to be patched and fixed isn't because the companies who are making the patches are making money off of them. It's actually kind of counter that. It's a huge issue for a company's brand (yes, PR and Marketing) when their software is the main reason most of the Internet or Corporate America goes down. Think of the damage control a company like Microsoft has to do when there is a massive worm spreading around the Internet like CodeRed or the Melissa Virus. It's huge. People change platforms, they decide they can no longer trust a company with such glaring vulnerabilities. They "switch." I myself started using Macs simply because I trust Unix way more than I do Microsoft's ability to protect my system e-mails and webpages.

Here's the problem though, those vulnerabilities usually aren't because some crazy hacker on a mission has decided they're going to ruin one of these worldwide brands. It's usually because the company themselves have someone, either on their payroll or contracted who has provided the notification of the exploit initially (internally or through a provider channel), either when they were working on the code, they crashed their own systems, or they had a hunch and tested their theory. They notify the company who in-turn rolls out a patch... these people are paid to provide this service.

People read everything with their own filter on the world. If they are a good person, when they see a patch, they probably think to themselves... I need to apply this because I don't want any downtime... but what if the people are bad? Okay, let's not say "good" and "bad" because that's not necessarily the case at all (and part of a larger discussion). Let's say they are users and then those other people who have "too much time on their hands" at the moment. I say this because at one point or another in a white hat hacker's life they have more than likely infected something or spread something on accident. They're not bad people, but if it's uncontrolled it could do just as much damage. Always test on an offline machine if you're going to open Pandora's Box.

The bad
So back to my point about the people filtering what they see. When someone who 1.) wants to experiment, 2.) has downtime, and 3.) a need of an idea for something to hack, they have this great expanse of information (the Internet)... I know it's pretty obvious right? Although [most] people think that most hackers all go to secret websites and have a secret handshake, that's really just the people who go to Defcon or who have friends who are hackers because they do it for a living, or they want to pretend they're hackers. Most of the other hackers I've met happened by accident because someone else mentioned that I hacked, then we talked about the level of what we were into.

Usually self-proclaimed "hackers" in my experience are in actuality script kiddies (people who use a program or a tool in a way they've read about to purposely cause chaos), so often when I'm confronted with the questions of what I do, I kind of go the other way and don't share what I'm into unless they let me know that they're "cool," A.K.A. not a script kiddie. Just like the branding issues companies have with being exploited, "hackers" white, black, and gray.... all hats, also have a branding issue because somewhere some [insert expletive here] is writing a virus that will cause harm and it says the same thing on my nametag to society that his says... I'm a creative professional with the means and ability. Society doesn't care whether I would do it or not or about my moral compass, but you have to think like a "bad guy" to outsmart a bad guy... it doesn't make me "bad." But it makes the unknowing populous marvel and wonder (in a bad way).

On with the Internet reference... when I say they have the Internet at their fingertips, they don't need to go to one of the heavily monitored websites for script kiddies or the IRC channels, all they have to do is browse through a company's patches. In the patches that most people install there is usually some bit of information that says what exploit or vulnerability is being patched. Apple doesn't share a lot of detail about this, but Microsoft usually tells you what they're patching if you follow enough links from Windows Update. Java, Ruby, PHP, and most other opensource languages will release it in a bugfix that you can read about. When it happened to Microsoft's brand before, Microsoft had already provided patches for the exploits for CodeRed and Melissa long before they were in the wild and running rampant. Most people however do not like applying patches because, just like going to the doctor, "If it ain't broke don't fix it."

I've heard all sorts of reasons why someone shouldn't patch something... "because if they don't know I'm running an older version I'm safe" or "it might bring down my machine so I wait a couple of months to test it." Zero-day patches just like zero-day exploits can also bring your machine to its knees. I wait about a week to make sure that a patch has been thoroughly tested by the masses. It takes most companies a couple of days to clean up after a failed patch, so that should be enough time to cover myself. (I can't afford to have downtime.)

The incurable
When an idle mind sees a patch and decides to take it upon themselves to figure out how to exploit it, then that's where the problems arise. The problems where there is a vulnerability that a company doesn't know about the day it's unleashed are called a zero-day exploits or holes and they're usually compromised in a zero-day attack... because the company has had zero days to prepare for the aftermath from a technical and marketing standpoint. These can be people purposely writing a virus or altering code and spreading it. Because a company has little or no warning then it can be catastrophic for the brand.

How is information sharing bad?
The problem is with the channels where information is shared. Most of the highly technical details about a vulnerability do not need to be out in the wild so a passing bot or web crawling search-engine can find them. They need to be behind at least one level of authentication. This makes is more like a deterrent because only the people who would really need to know about something would take the time and effort to go in and look at all of the specifics. Potentially harmful individuals might go in and still compromise a machine or series of systems, but a casual passer-by wouldn't see the info to get any ideas.

Really observant individuals might actually take the time to find a pattern in [poor] programming. For instance Microsoft has been pretty bad about securing Internet Explorer and the way it is interconnected with their operating systems. In the past, when someone logged in with the default Administrator account they could open an e-mail or a webpage and take down their machine with full privileges. Luckily it's a little more difficult for most users now.

On another note Whitepapers can be something of a major problem as well. I downloaded yet another Whitepaper on SQL Injection attacks again today. Nothing new or earth shattering, but it always pays to look to see what I might be up against. I'm always interested in new perspectives.

A thief who can see a whole building and examine it in full detail, might realize it's much easier for them to drive through the wall and bypass the door and window sensors on the alarm system altogether. The same thing applies to Whitepapers and Patch Descriptions on the web. Although much of the media clambers for information about the technical specifics of what happened, it's probably safer if all of that detail isn't on the record completely and in the open. PR and Marketing departments should be the main filter in brand protection. After all, too much self-provided information might actually help in destroying your brand. (Same goes for real hackers.)

Soap Box
If you have a person (or group of people) in your organization or company who really want to support some of the open-source platforms like Java, PHP, Ruby-on-Rails, and so forth they also need to understand the responsibilities that come with maintaining an effectively secure system. Everything needs to be patched and it needs to stay somewhat up-to-date. When companies invest in new ideas and those ideas fail the people who are working on the front lines and in the trenches are the ones that are hit. Most companies can reboot from a failed experiment, but most people can't.

That's all for now.