Logistic regression, test out LaTeX

In industrial products, logistic regression/maximum entropy classifier is often preferred than SVM. But if not well regularized, MaxEnt will have severe overfitting problem. In his famous textbook, Chris Bishop alludes to this problem without details.

The objective function of MaxEnt is:

$p( {\bf t} | {\bf w}) = \prod_{n=1}^{N} y_n^{t_n} \{1-y_n\}^{1-t_n}$

Writing something down is a good exercise

Broadly speaking, there are seven modes in learning: reading, listening, watching, thinking, talking, writing, visualizing.

Reading, listening, and watching are three input modes. Taking input is relatively easy: you just grab a book, or plug in your ear pods, or open up yet another video, and lay comfortably in the sofa. You may or may not focus your mind on the material; even if you do, your thoughts are mainly dependent, guided by the material, like a kid following the teacher. Unless there’s an explicit need (make an argument/bargaining/defense), the brain is swayed from side to side. In other words, input modes are System 1-friendly.

Thinking is the bridge from input modes to output modes. It can be mobilized by immediate inputs, or come up spontaneously after a long “fermentation”.

The output, or talking, writing, and visualizing are not so “cozy” since they are to produce information. If you want to “make sense”, or expect other people to understand your thoughts, your job is non-trivial. These “other people” may include the producer himself, as he wants to keep a record for later use. To state it with limited symbols, the producer has to first know the audience, then be concise, logical, even passionate and humorous. These are not innate skills; System 2 is often the player here.

And of course there’s the loop of interaction. Information flows from the producer to the consumer, synchronously or asynchronously, by air or wire, understood or (usually) misinterpreted, stirs thoughts in the consumer. If the consumer responses, the 2nd round kicks off.

Throughout civilization, writing is the most common output mode. Of course nothing prevents one from uploading his speech, photo, video, sketch or other products of output, other than text. But I’d say text is still among the most expressive choices.

Hence the title of the first blog.