Multilayer perceptrons and fractals

C. A. Murthy, Jennifer Pittman

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

In this article, a mathematical relationship between the gradient descent technique and contractive maps is examined. This relationship is based upon the observation that the convergence of the gradient descent technique can be proved using results in fractal theory - more specifically, results concerning contractive maps - as opposed to results based on Taylor's series. This proof, involving the eigenvalues of the Hessian matrix of the gradient descent technique's objective function, is presented. A simple example is given in which steps from the aforementioned proof are used to find conditions under which a specific multilayer perceptron is guaranteed to converge. Since the gradient descent technique is used in multilayer perceptrons, and contractive maps give rise to fractals, a theoretical relationship is thus established between multilayer perceptrons and fractals.

Original languageEnglish (US)
Pages (from-to)137-150
Number of pages14
JournalInformation Sciences
Volume112
Issue number1-4
DOIs
StatePublished - Jan 1 1998

Fingerprint

Gradient Descent
Multilayer neural networks
Perceptron
Fractals
Multilayer
Fractal
Taylor series
Hessian matrix
Objective function
Eigenvalue
Converge
Gradient
Relationships

Keywords

  • Attractor
  • Contractive map
  • Fixed point
  • Fractal
  • Gradient descent
  • Hessian matrix

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Theoretical Computer Science
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Cite this

Multilayer perceptrons and fractals. / Murthy, C. A.; Pittman, Jennifer.

In: Information Sciences, Vol. 112, No. 1-4, 01.01.1998, p. 137-150.

Research output: Contribution to journalArticle

Murthy, C. A. ; Pittman, Jennifer. / Multilayer perceptrons and fractals. In: Information Sciences. 1998 ; Vol. 112, No. 1-4. pp. 137-150.
@article{af69499183534bf3bc25b8e0294fde47,
title = "Multilayer perceptrons and fractals",
abstract = "In this article, a mathematical relationship between the gradient descent technique and contractive maps is examined. This relationship is based upon the observation that the convergence of the gradient descent technique can be proved using results in fractal theory - more specifically, results concerning contractive maps - as opposed to results based on Taylor's series. This proof, involving the eigenvalues of the Hessian matrix of the gradient descent technique's objective function, is presented. A simple example is given in which steps from the aforementioned proof are used to find conditions under which a specific multilayer perceptron is guaranteed to converge. Since the gradient descent technique is used in multilayer perceptrons, and contractive maps give rise to fractals, a theoretical relationship is thus established between multilayer perceptrons and fractals.",
keywords = "Attractor, Contractive map, Fixed point, Fractal, Gradient descent, Hessian matrix",
author = "Murthy, {C. A.} and Jennifer Pittman",
year = "1998",
month = "1",
day = "1",
doi = "10.1016/S0020-0255(98)10028-2",
language = "English (US)",
volume = "112",
pages = "137--150",
journal = "Information Sciences",
issn = "0020-0255",
publisher = "Elsevier Inc.",
number = "1-4",

}

TY - JOUR

T1 - Multilayer perceptrons and fractals

AU - Murthy, C. A.

AU - Pittman, Jennifer

PY - 1998/1/1

Y1 - 1998/1/1

N2 - In this article, a mathematical relationship between the gradient descent technique and contractive maps is examined. This relationship is based upon the observation that the convergence of the gradient descent technique can be proved using results in fractal theory - more specifically, results concerning contractive maps - as opposed to results based on Taylor's series. This proof, involving the eigenvalues of the Hessian matrix of the gradient descent technique's objective function, is presented. A simple example is given in which steps from the aforementioned proof are used to find conditions under which a specific multilayer perceptron is guaranteed to converge. Since the gradient descent technique is used in multilayer perceptrons, and contractive maps give rise to fractals, a theoretical relationship is thus established between multilayer perceptrons and fractals.

AB - In this article, a mathematical relationship between the gradient descent technique and contractive maps is examined. This relationship is based upon the observation that the convergence of the gradient descent technique can be proved using results in fractal theory - more specifically, results concerning contractive maps - as opposed to results based on Taylor's series. This proof, involving the eigenvalues of the Hessian matrix of the gradient descent technique's objective function, is presented. A simple example is given in which steps from the aforementioned proof are used to find conditions under which a specific multilayer perceptron is guaranteed to converge. Since the gradient descent technique is used in multilayer perceptrons, and contractive maps give rise to fractals, a theoretical relationship is thus established between multilayer perceptrons and fractals.

KW - Attractor

KW - Contractive map

KW - Fixed point

KW - Fractal

KW - Gradient descent

KW - Hessian matrix

UR - http://www.scopus.com/inward/record.url?scp=0032293057&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032293057&partnerID=8YFLogxK

U2 - 10.1016/S0020-0255(98)10028-2

DO - 10.1016/S0020-0255(98)10028-2

M3 - Article

AN - SCOPUS:0032293057

VL - 112

SP - 137

EP - 150

JO - Information Sciences

JF - Information Sciences

SN - 0020-0255

IS - 1-4

ER -