Universal Approximation Theorem


The Universal Approximation Theorem states that:

Let φ(·) be a nonconstant, bounded, and monotonically-increasing continuous function. Let Im0 denote the m0-dimensional unit hypercube [0,1]m0. The space of continuous functions on Im0 is denoted by C(Im0). Then, given any function f Э C(Im0) and є > 0, there exist an integer m1 and sets of real constants αi, bi and wij, where i = 1, …, m1 and j = 1, …, m0 such that we may define:

  F( x_1 , dots, x_{m_0} ) =
  sum_{i=1}^{m_1} alpha_i varphi left( sum_{j=1}^{m_0} w_{i,j} x_j + b_iright)

as an approximate realization of the function f; that is,

  | F( x_1 , dots, x_{m_0} ) - f ( x_1 , dots, x_{m_0} ) | < varepsilon

for all x1, x2, …, xm0 that lie in the input space.

[ From Wikipedia ]

Beautiful, isn’t? 😛

Leave a Reply

Your email address will not be published. Required fields are marked *