- Ancient_History (31)
- Artificial General Intelligence (7)
- Business_Ideas (17)
- Expatriation (14)
- Links (16)
- Machine Learning (Narrow Artificial Intelligence) (78)
- Mathematics (62)
- Miscellaneous (25)
- Music (15)
- Paleolithic_Lifestyle (26)
- Photos (2)
- Political_Economy (774)
- Reading (16)
- Science_Technology (228)
- Shale_Oil_Gas (15)
- Sports (7)
Follow me on TwitterMy Tweets
Follow me on Facebook
The format problems were due to stray end tags /div in the post “Hong Kong’s Remarkable Fiscal Policy by Dan Mitchell“. Specifically, they occurred in the a copy of a jpg from Mitchell’s article. When I deleted it, the format problems vanished.
The media hype surrounding artificial neural networks (ANN) is becoming absurd. These articles tend to be uncritical summaries of company press releases. The most over the top articles are those derived from press releases of companies seeking funding.
ANNs are useful machine learning algorithms. They perform reasonably well in static image recognition, some types of pattern recognition, and other tasks. However, they are far too simple in how they model time, feedback, etc. Also, they are very inefficient. Companies such as Google, Baidu, and Microsoft have used enormous numbers of servers to accomplish image recognition and language translation via ANNs. Indeed, the recent surge in so-called deep ANNs is largely due to better and cheaper computing power and some clever algorithmic tricks. As a machine learning algorithm, ANNs have been very successful and will continue to be a useful tool for a growing number of problems in an ever increasing number of areas.
However, the inherent limitations of ANNs will prevent them from every even approaching artificial general intelligence (AGI; see “Artificial General Intelligence Versus Narrow Artificial Intelligence (Machine Learning)“) and will limit their use in machine learning. While much of the hoopla about ANNs has been about their success in image recognition, it has been realized that there are problems lurking. In “The Flaw Lurking In Every Deep Neural Net by Mike James“, I linked to an article that cites a paper demonstrating two problems with ANNs.
On the NuPIC mailing list, Jeff Hawkins made some interesting comments about the limitations of ANNs.
‘Thanks for finding this. It is very interesting. I haven’t read the original paper yet, but I have preliminary thoughts that might be worth sharing.
The first problem they point out is that low level DL (deep learning) neurons are no better than random neurons. This sounds remarkably like our experience using an untrained spatial pooling function, what we call a “random SP”. We have found that using an untrained spatial pooling function works remarkably well. So well that we often don’t train the SP synapses until late in the development process. Spatial pooling is a mapping process and a random SP does the mapping just fine. The problem with untrained spatial pooling is that small changes in the input can lead to a large change in the output. This is what they are reporting in the second problem in the paper. A properly trained SP function is better than random and does not exhibit the “small change in input causes a large change in output” problem.
I don’t know why the DL neurons are not learning to be better than random. I don’t have a good enough insight into exactly how DL works to say. But we do understand a very similar phenomenon in HTM networks and we know that a properly trained HTM network does not have this issue.
On a more broader analysis we know that DL networks are not at all like the neural networks in biological brains. Anyone who says that ANNs and DL networks are similar to biological neural networks doesn’t know much about brains. The biological and DL networks are so different that the brittleness they are seeing almost certainly does not exist in biological brains.
In a couple of recent talks I have started to point out the difference between ANNs and biological networks to drive home that HTM networks are much closer to biology.
- biological and HTM neurons have active distal dendrites, ANN neurons don’t
– biological and HTM neurons have thousands of synapses, typical ANN neurons have dozens
– biological and HTM neurons have unreliable, low precision, synapses, most ANN neurons rely on synaptic weight precision
– biological and HTM neurons learn mostly by forming new synapses, ANN neurons only learn by synaptic weight modification
A typical biological neuron has 100’s of proximal synapses (say 500) and these sum linearly at the soma. The proximal synapses represent only 10% of the synapses on a cell. They correspond most closely to the dozens of synapses on ANN neurons. However, there is a huge difference in how biological neurons use their proximal synapses. The proximal synapses on a biological neuron recognize dozens of unique feedforward patterns, not one. In HTM theory this is how Temporal Pooling is implemented. A cell learns to respond to dozens of unique feedforward input patterns. Temporal pooling is an absolute requirement for inference and every neuron is doing it. For a cell to recognize multiple unique patterns on its proximal dendrites requires that the patterns to be recognized are sparse. So the entire process can only be understood in the context of SDRs.
Again, I am not an expert on deep learning but I have yet to meet a DL expert who understands these concepts or the relevant biology. I am certain that HTM networks applied to applications like vision will not exhibit the problems talked about in this paper. Brains won’t either.‘
Note that SDR stands for sparse distributed representation and HTM stands for hierarchical temporal memory.
ANNs and Hawkins’ HTM are both useful machine learning technologies that perform useful tasks, as is true of many other algorithms. None of them have reached the point to be considered universal algorithms that outperform all others on all problems. Thus, it is up to individuals to select and implement the best algorithm for a specific task.
A few days ago, the widgets on the right side of all pages were no longer visible on the homepage. They remained visible on the About and Libertarian Fiction pages and for individual posts. The only fix that I have been able to discover is that of publishing a blank post. I posted my problem to the wordpress.org forums, so if someone can suggest another way to fix the problem without the blank post, I will implement it.
“What Is a Liberal?” is the title of Henry Hazlitt’s Newsweek column from December 5, 1955. In this article, Hazlitt notes the perversion of the word liberal from designating advocates of freedom to worshipers of the state.
Although this is a overall a good article, I cannot praise the odd first paragraph in which Hazlitt lauds the utterances of federal judge. While Hazlitt was generally a passionate advocate of liberty, he erred in his advocacy of the Cold War and nationalism. Regardless, his remarks about the perversion of language by statists are important and still relevant today.
‘We have all been long indebted to Judge Harold R.
Medina, now of the U.S. Circuit Court of Appeals,
for the patience, balance, and firmness with which
he presided at the trial of eleven top Communists in
1949. A few weeks ago he put us further in his debt by
exposing the inverted semantics into which most of us
have recently fallen in our political discussion—particularly
in the strange use of the word “liberal.” He
regards himself as a liberal, Judge Medina declares, and
he does not intend “to be frightened away because the
Communists and their coadjutors have tried to appropriate
the word ‘liberal’ just as they have the names of
our great Presidents, Abraham Lincoln and Thomas
Jefferson, by the use of Aesopian language, twisting
names and personalities to suit their purpose.”
Today Socialists, fellow travelers, and Communists
all call themselves liberals. As a result, as Judge Medina
points out, the word has taken on “a sinister and evil
connotation.” I should like to supplement his own
remarks on this strange usage and its results.
Originally (as its Latin root liber implies) the liberal
was a man who believed in freedom. The foundations
of the liberal tradition were laid in England by such
great figures as Milton, Locke, Hume, Burke, Adam
Smith, and Mill. Politically, the liberal tradition stood
for freedom for the individual, the Rule of Law, strict
limitation of the powers of government, and decentralization
and diffusion even of these limited powers.
Economically, the liberal tradition stood for protection
of private property, and for freedom of trade, of prices,
of markets, and of enterprise.
But through historical accident and intellectual
confusion, the word “liberal,” particularly in this country,
altered and finally reversed its meaning. Today, in
popular speech, a “liberal” has come to mean a person
who wants constantly to expand the powers of government,
and to centralize them in Washington at the
expense of the states and localities. It has come to mean
a person who disparages Congressional restraint on the
executive power and who prefers bureaucratic discretion
to the Rule of Law. Economically, it has come to
mean a person who distrusts freedom of markets and
freedom of enterprise; who distrusts private ownership
and management and extols government ownership and
management; who presses for more government “planning,”
and who wants to tax and penalize success in
order to subsidize failure. Continue reading
“Be it or be it not true that Man is shapen in iniquity and conceived in sin, it is unquestionably true that Government is begotten of aggression, and by aggression.” – Herbert Spencer. H/T Dan Sanchez.
“Reading the Road Map to a Police State” is a review of Radley Balko’s Rise of the Warrior Cop: The Militarization of America’s Police Forces. Balko’s book was published last year but the paperback version was released recently. Tao provides a good summary of Balko’s arguments regarding how the US came to resemble an occupied nation.
‘If there was any silver lining to the horrifying events that took place in Ferguson, Missouri which riled the month of August, it has finally brought the issue of police militarization to the forefront. As outrageous as the police shooting death of unarmed 18-year-old Michael Brown was, the brutal law enforcement response in the form of running roughshod over the First Amendment and resorting to quasi-martial law to mostly peaceful protests by local residents and activists was worse. To many observers, what took place in a Midwest suburb was indistinguishable from scenes out of occupied Iraq.
How did this happen? For an answer, the writings of investigative journalist Radley Balko are an invaluable resource. Perhaps more than any other person, Balko has reported substantially on police militarization and injustice across the country for years.
The full details can be found in his book Rise of the Warrior Cop: The Militarization of America’s Police Forces . This important book, which was recently released in its paperback edition, could not have arrived at a better time. Despite going into an intellectually rigorous analysis of law, politics, and history, Balko has a gift for storytelling, which highlights many heartbreaking stories and makes Rise of the Warrior Cop an accessible and gripping read.
In the introduction, Balko begins with the provocative question:
How did we evolve from a country whose founding statesmen were adamant about the dangers of armed, standing government forces — a country that enshrined the Fourth Amendment in the Bill of Rights and revered and protected the age-old notion that the home is a place of privacy and sanctuary — to a country where it has become acceptable for armed government agents dressed in battle garb to storm private homes in the middle of the night — not to apprehend violent fugitives or thwart terrorist attacks, but to enforce laws against nonviolent, consensual activities?
In the first chapter, Balko traces classical history and its lessons on America’s Founders as well their own experiences under British rule. As students of the Enlightenment, they were familiar with how the Roman Republic was overthrown by ambitious military leaders and how the Praetorian Guard in the Empire era, which not only acted as bodyguards for the emperor but also took on many policing roles as well, was responsible for much political intrigue and instability. In the lead-up to the American Revolution, British authorities used the hated writs of assistance to enforce tax laws and to crackdown on contraband in the colonies. This type of general warrant allowed for authorities to “search broad groups of people, for evidence of any number of crimes, sometimes over long stretches of time.” As bad as they were, Balko noted that in contrast to what police can do today, the writs of assistance could not be exercised at night and they required a knock-and-announcement before entry into a private home. Finally, it was the deployment of British soldiers to enforce the law that brought long-simmering tensions to a boil. After the Revolutionary War, with these abuses still fresh on their minds, the Founders framed and ratified a Constitution with a Bill of Rights.
The Fourth Amendment, in particular, was written explicitly to prohibit general warrants and to reinforce the Castle Doctrine, an even older principle carried over from the British common law that can be traced back to antiquity. The Castle Doctrine simply reinforces the timeless idea that “a man’s home is his castle.” As explained by Balko:
Implicit in the sentiment is not only the right to repel criminal intruders but also the idea that the state is permitted to violate the home’s sanctity only under limited circumstances, only as a last resort, and only under conditions that protect the threshold from unnecessary violence. Thus, before entering without permission, government agents must knock, announce and identify themselves, state their purpose, and give the occupants the opportunity to let them in peacefully. … The announcement requirement under English law was not a formality, as it has become in police raids today. It was elemental. Its purpose was to give the homeowner the opportunity to avoid violence, distress, and the destruction of his property.‘
Dorobăț wrote a good article about how the Hanseatic League developed an extensive trading network via equity rather than debt.
‘In fact, international trade thrived in times when banking and financial systems were not only in their in their infancy, but also when oceanic trade was dangerous and expensive. Nonetheless, it began as a self-financed venture, a venture for which merchants themselves set money aside. The Hanseatic League (c. 13th to 17th century)—a commercial association of traders from German towns—is a good example of this type of financial behavior.
Philippe Dollinger’s detailed chronicle of Hansa’s development shows that trades were financed from a merchant’s own accounts, or from those of his associates. Entrepreneurs bought shares in each cargo and in each ship, and subsequent profits and losses were divided in proportion to the capital invested; the captains of the ships, who sailed together for mutual protection, sometimes joined the ranks of shareholders. In spreading their investments over several cargoes—diversifying their portfolios, as it were— merchants also reduced the risk of transporting their goods over long distances. For centuries, no banks took part in these commercial networks, “but this in no way precluded the existence of merchants operating on a large scale, investing large amounts of capital, carrying out … complex commercial operations in various geographic regions” (Dollinger 1970, 168).
The Hanseatic League’s approach to business was furthermore defined by an outright hostility to debt, as the practice of borrowing money was proscribed in many mercantile quarters. By the 14th century, Hanseatic towns embarked upon a systematic campaign against financing commerce via credit,
on the grounds that it caused instability of prices, which would upset business. Sometimes a buyer… not being obliged to pay on the nail, would agree to an excessive credit. Credit was also accused of increasing the temptation to take risks, and even worse, of favoring the dishonest schemes of unscrupulous merchants, thus compromising the good name of the Hansa (Dollinger 1970, 205).‘
An interactive map of the Hanseatic League can be found here.