Web logs

Boosting: In lieu of training all the models separately as in bagging, boosting trains models sequentially. Each new model is trained to correct the errors made by the previous ones. The first tree is examined and the weights of those observations that are hard to classify are increased and the weights for those that are easy to classify are reduced. This modified data is used to build the next tree. This process is repeated for a defined number of iterations. Predictions of the final ensemble model is therefore the weighted sum of the predictions made by the previous tree models. GBM uses loss functions Since each tree is fit to residuals as against the original output parameter, each tree is small and improves prediction in the parts where prediction is bad. All the models might make the same mistake in the standard ensemble method. Compute error by deducting forecasted value from target value (e1= y - y1_forecasted) Build a new model on errors (e1_forecasted) as target variabl...

Regularisation is used to constrain the model to fewer degrees of freedom or regularise beta estimates towards zero in order to avoid overfitting. Hence it avoids a complex or flexible model. Regularisation reduces the variance of the model considerably without substantial increase in its bias. λ is the tuning parameter used to penalise the flexibility of the model.  As the value of λ increases, it reduces the value of beta estimates except for intercept and thus reducing the variance. However beyond a certain threshold, the bias starts increasing as the model starts losing important information resulting in under fitting. Note that Regularization adds penalty to the higher terms and their importance reduces. Lasso uses modulus of Beta estimates to penalize and this is known as L1 norm while Ridge uses squares of Beta estimates to penalize and this is known as L2 norm. Lasso can penalise some of the beta estimates to be equal to zero when λ is large enough resulting in featu...

Bias refers to the deviation of the predicted values from the correct value. The error occurs when you make wrong assumptions about data. In other words, it is the error that is created when you represent a real-life complex problem using a simpler model while it might be making them easier to understand. For instance, when you build a linear model to solve for a non-linear problem. It results in under fitting and makes them less flexible. Parametric algos like Linear Regression can produce high bias while non-parametric algos like Decision Trees make good assumptions about the training data and target function and hence do not have high bias. Variance refers to the change that occurs when the model is applied on a different training data. It occurs when the model captures not just the underlying pattern but noise as well. It results in overfitting. In other words, it is memorising the data. It is often observed in Decision Trees. When the observations are limited but the...

In his book 'An Enquiry concerning Human Understanding'  David Hume posited that inherently fallible evidence is insufficient proof against natural laws - eyewitness testimony can’t prove a miracle. Bayes, a Presbytarian minister, motivated to rebut him was interested in understanding how much evidence would we need to be persuaded that something is a probability regardless of how improbable it is. He developed an equation that focuses on updating our beliefs with new evidence. His work, An Essay towards solving a Problem in the Doctrine of Chances, was reviewed by Richard Price after Bayes' death who believed that Bayes’ Theorem helped prove the existence of God. When we are presented with new information, we can use Bayes’ Theorem to refine our pre-existing belief.  It is fairly easy to determine the probability of the effect given a cause. Since Bayes’ Theorem works in the reverse direction, many find it complicated.

The green line has a negative Beta and the black one has a positive beta for logistic regression.

we are now at a critical juncture where many of the systems we need to master are fiendishly complex, from climate change to macroeconomic issues to Alzheimer’s disease. The problem is that these challenges are so complex that even the world’s top scientists, clinicians and engineers can struggle to master all the intricacies necessary to make the breakthroughs required. It has been said that Leonardo da Vinci was perhaps the last person to have lived who understood the entire breadth of knowledge of their age. Since then we’ve had to specialise, and today it takes a lifetime to completely master even a single field such as astrophysics or quantum mechanics. The systems we now seek to understand are underpinned by a vast amount of data, usually highly dynamic, non-linear and with emergent properties that make it incredibly hard to find the structure and connections to reveal the insights hidden therein.  Kepler and Newton could write equations to describe the motion of p...

In  statistics , the  generalized linear model  ( GLM ) is a flexible generalization of ordinary linear regression  that allows for response variables that have other than a  normal distribution . The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a  link function  and by allowing the magnitude of the variance of each measurement to be a function of its predicted value.  in many cases when the response variable must be positive and can vary over a wide scale, constant input changes lead to geometrically varying rather than constantly varying output changes

In  statistics  and, in particular, in the fitting of  linear regression  models, the  elastic net  is a regularized regression method that combines the L1 and L2 penalties of the  lasso  and  ridge  methods.

Lasso  a regularization technique that's useful for feature selection and to prevent over-fitting training data. It works by penalizing the sum of absolute value (L1 norm) of weights found by the  regression.  

Want something that is potentially human comprehensible? Use decision trees or rules. Have a situation where you have lots of memory, but have to learn incrementally and evaluate quickly? Use Nearest Neighbour. Have a clear binary decision in a continuous space? Use SVMs. Have thousands of independent attributes and lots of data? Use Naive Bayes. Have a situation where you know which attributes are correlated with which? Use Bayes nets.

ML algorithms are an evolution over normal algorithms. They make your programs "smarter", by allowing them to automatically learn from the data you provide. You take a randomly selected specimen of mangoes from the market ( training data ), make a table of all the physical characteristics of each mango, like color, size, shape, grown in which part of the country, sold by which vendor, etc ( features ), along with the sweetness, juicyness, ripeness of that mango ( output variables ). You feed this data to the machine learning algorithm ( classification/regressio n ), and it learns a model of the correlation between an average mango's physical characteristics, and its quality.  Next time you go to the market, you measure the characteristics of the mangoes on sale ( test data ), and feed it to the ML algorithm. It will use the model computed earlier to predict which mangoes are sweet, ripe and/or juicy. The algorithm may internally use rules similar to the rules y...

Credit cards with rewards generally target people who spend a lot on credit cards. That high transaction volume is what makes it viable for issuers to provide the rewards. Rewards  are drawn from the in terchange revenue that issuers get from merchants.    To sup port those high transaction volumes, rewards cards generally have a high credit limit. Lower limits do not make economic sense. These cards are generally offered to people with very good credit as a result.    People with no credit history are generally (but not always) considered to be higher risk by issuers, which translates to lower credit limits, which squeezes out rewards propositions. 

While there are generic FICO scores used for all types of lending,  there also are FICO scores for specific types of lending designed for mortgage lending and insurance purposes. Further, there are FICO scores for specific types of lenders, such as credit unions or traditional banks. Adding to that, there are FICO credit scores created for specific lenders, called custom scores. FICO is a good company, and it produces good credit scores. But it also has competitors, including Scorex, a credit scoring company owned by Experian, that also produce credit scores in much the same way. There are also lenders that produce their own credit scoring systems

The way to solve this is to save the customer’s card number, verify  that it’s legitimate, and then charge all the customers once the deal’s  critical mass has been achieved. However, storing card numbers is a pain because if the card numbers  are stolen from your servers you can face fines which can easily exceed  $millions of dollars for a large breach.  Also, to get a a merchant account with a credit card processor you will need to sign a document telling them that you are PCI compliant.  Being PCI compliant ( http://feefighters.com/b log/easy... ) is pretty easy if you don’t store card numbers, but if you do wish to store card numbers it requires security audits. You can get most of the benefits of storing card numbers locally without worrying too much about PCI by using tokenization ( http://feefighters.com/b log/cred... ). You also have 2 options with respect to verifying the card number before storing it as a token. Do a quick algorithmic check us...

American Express has always charged higher merchant interchange rates than Visa, MasterCard and Discover.  They justify these rates to merchants by claiming to have a higher end card holder base that spends more money with retailers and is, therefore, worth more to the retailer. PayPal pioneered the simple blended interchange cost structure for merchant accounts.  The theory was that merchant fees were excessively complicated and unfriendly to merchants, with different rates for different categories of goods of services or different card types, etc.   PayPal took the merchant friendly route and offered merchants a simple to understand blended fee structure.  It is a much more merchant friendly form of pricing, hands down. The problem with PayPal's simple blended average pricing for Amex was that it undercut Amex's pricing in the market.  It became cheaper to accept Amex through your PayPal account than it was to go direct to Amex.  This was obviou...

Every credit card rewards program is structured differently.   You receive a cashback bonus of 1%, 2%, or 5% depending on where you shop. These rewards are generally thought of as being funded by a component of the fees that a merchant has to pay to accept credit cards calle d interchange or merchant discount. This fee can be fixed or variable and can vary based on the merchants' negotiated agreement with an acquiring bank/credit card processor, with a nationwide average of 1.79%. Usually it is the sponsor of the program who  pays for credit card bonus points , that could be either the bank behind the credit card (in case the card is not affiliated to a brand) or the co-branding company (such as the store or airline, etc). Fraction of the revenue is deferred until the points are redeemed in the future. The expense associated with the program are under the marketing budget.

(1)  The Merchant POS is the in-st ore scanner or shop ping cart. Often they will help to do the initial collection of  payment data. Online examples include Shopify, Magento, Drupal, and Big Platform. The Brick and Mortar POS landscape is dominated by NCR, IBM, HP, and Verifone - and Square is quickly emerging. For the most part, merchants hold an 'Approved Merchant Account (Token)' however many e-commerce platforms are now providing this as a service. Fees are usually based on a monthly fixed cost, a per-transaction fee, and an overall percent of transaction fee. Generally the POS (or sometimes Gateway) charges the merchant a single fee that covers all the downstream costs. (2)  The payment gateway is a secure connection between a POS and Processor. Historically, processor connections required such high security that it was not economical for a POS or merchan t to implement themselves. Of late, these costs have come down and we now see the integration of POS, Gateway...

(1)A customer completes a transaction at the merchant POS. POS system then passes the transaction, card, and merchant bank info to the Gateway (2) The Payment Gateway collects transaction, card, and merchant data from the Gateway and passes it securely to the linked Payment Processor (3) The Payment Processor collects the Gateway data and identifies the correct Network/Association based on the card type. Processor then routes all the transaction data to the Network. (4) The Network collects the transaction information and verifies the available balance with the Card Issuer*. In addition to the balance check, the network performs a security check to ensure the card is within normal spending patterns. If the transaction is approved by Card Issuer and passes the security check, the Network passes an affirmative response upstream to the POS and downstream to Card Issuer**. During Settlement***, network collects transaction amount from Card Issuer account and passes it to the Merchant Acco...

Firstly,  Square (product)  is not a replacement but a supplement to credit card payments and therefore it is still part of the traditional credit card payment system. Square enables users with iPhone, iPod Touch or iPad to easily accept payments and removes the hassles of signing up for a merchant account, buying a point of sale terminal and credit card reader. After you sign up with Square [ www.squareup.com ] they send you a free credit card reader. You will then need to download the Square app [ http://itunes.apple.com/ us/app/s...   from the  App Store , open the app and plug the Square credit card reader in the headphone jack to accept credit card payments. Square charges 2.75% towards transaction fees. Banks and payment processing service providers that offer merchant solutions usually charge a flat fee and varying percentage (1% to 5%) of the transaction value in addition to a monthly fee. Square does not charge flat fee or monthly fees. ...

If  you are interested in marketing credit cards with your brand, big banks have programs for issuing "affinity" and "co-branded" credit cards.  Examples include cards that are branded by charities (e.g., Nature Conservancy), colleges/universities, consumer brands (e.g., MLB baseball teams).  The issuers then pay the affinity groups or brands for the use of the brand and often a fee for each new account.  Larger brands often work with banks to create reward credit cards that offer rewards connected with credit card (e.g. airline credit cards with airline miles).   Chase Paymentech is Square’s acquirer. So Square pays Chase Paymentech (subsidiary of JP Morgan, who is the de-facto Acquiring Bank) for gateway fees, processing fees, card scheme fees, and interchange fees. Square's business model is very much like Paypal's business model but probably with lesser risks because Square accepts card-present payments and one would likely see less charge...