Rényi entropy measure of noise-aided information transmission in a binary channel

François Chapeau-Blondeau, David Rousseau, and Agnès DelahaiesLaboratoire d’Ingénierie des Systèmes Automatisés (LISA), Université d’Angers, 62 Avenue Notre Dame du Lac, 49000 Angers, France

�Received 8 February 2010; published 12 May 2010�

This paper analyzes a binary channel by means of information measures based on the Rényi entropy. Theanalysis extends, and contains as a special case, the classic reference model of binary information transmissionbased on the Shannon entropy measure. The extended model is used to investigate further possibilities andproperties of stochastic resonance or noise-aided information transmission. The results demonstrate that sto-chastic resonance occurs in the information channel and is registered by the Rényi entropy measures at anyfinite order, including the Shannon order. Furthermore, in definite conditions, when seeking the Rényi infor-mation measures that best exploit stochastic resonance, then nontrivial orders differing from the Shannon caseusually emerge. In this way, through binary information transmission, stochastic resonance identifies optimalRényi measures of information differing from the classic Shannon measure. A confrontation of the quantitativeinformation measures with visual perception is also proposed in an experiment of noise-aided binary imagetransmission.

DOI: 10.1103/PhysRevE.81.051112 PACS number�s�: 05.40.�a, 02.50.�r

I. INTRODUCTION

Information measures are important both for the founda-tion of information sciences and for practical applications ofinformation processing. For measuring information, a gen-eral approach is provided in a statistical framework based onthe information entropy introduced by Shannon �1�. As ameasure of information, the Shannon entropy satisfies somedesirable axiomatic requirements and also it can be assignedoperational significance in important practical problems, forinstance, in coding and telecommunication �2�. For informa-tion measure, an extension to the Shannon entropy has beenproposed with the Rényi entropy �3�. As an extension, theRényi entropy satisfies a set of axiomatic requirements �3�,slightly less restrictive than the Shannon case, but still pre-serving suitability as a natural measure of information. TheRényi entropy offers a parametric family of measures, fromwhich the Shannon entropy is accessible as a special case.On the operational side, it is appropriate to test the Rényientropy as a general parametric approach for the measure ofinformation in concrete practical processes. In this respect, itis the special Shannon case of the measure that has beenmost largely involved in practice. Beyond, it remains inter-esting and useful to identify and investigate the possibilitiesof assigning some specific role, significance, or properties tothe Rényi entropy outside the special Shannon case. In thisdirection, the present paper proposes an analysis, withRényi-entropy-based measure of information, of a basic pro-cess of information transmission with a binary channel. Thisanalysis is a natural extension of, and contains as a specialcase, the classic reference model of information transmissionformed by the binary channel quantified with Shannon-entropy-based measure �2�. In addition, in the analysis of thebinary channel based on the Rényi entropy, we will specifi-cally examine the possibility of a stochastic resonance phe-nomenon or a regime of information transmission aided bynoise.

Stochastic resonance, in a broad acceptation, designatessituations where the noise can play a constructive role in

transmission or processing of information �4–6�. Stochasticresonance has been reported to operate in many differentareas, including electronic circuits �7–9�, neuronal processes�10–13�, nonlinear sensors �14–16�, and optical devices�17–19�. For example, the currently vibrant state of the topicof stochastic resonance in relation to neuroscience can beillustrated in recent works �20–24�. Stochastic resonance isusually identified by a relevant measure of performancewhich culminates �resonates� at a maximum for a nonzerooptimal amount of noise. Stochastic resonance was first dem-onstrated in the transmission of a periodic signal aided bynoise. In such situation, the standard measure of performanceis a signal-to-noise ratio defined in the frequency domain�4,25,26�. Later, stochastic resonance was extended to noise-aided transmission of aperiodic signals, assessed by cross-correlation measures �27–29�. Situations of noise-aided sig-nal detection or estimation were also reported andcharacterized by appropriate performance measures, such asa probability of detection �5,30,31�, an estimation error�32–34�, or a Fisher information �35–37� improvable by thenoise. Owing to their status of general information measures,information-theoretic quantities based on the Shannon en-tropy were also employed to characterize stochastic reso-nance. Situations were reported of mutual information�38–41� or information capacity �42–44� improved by noise.A natural extension is to test the ability of general informa-tion measures based on the Rényi entropy for the character-ization of stochastic resonance. A study is proposed on thisissue in the present paper. We will demonstrate that Rényi-entropy-based measures are capable of registering noise-aided transmission in an information channel. Moreover,when seeking the information measures benefiting from thelargest amount of noise, we will show that nontrivial optimalorders emerge for the Rényi entropy, which differ from thespecial order defining the Shannon entropy.

A previous study appeared in �45� associating the Rényientropy and stochastic resonance. The problem addressed in�45� is, for a deterministic signal in additive noise, to esti-mate the signal-to-noise ratio classically defined as the ratioof the signal power to the noise power. For this purpose, Ref.

PHYSICAL REVIEW E 81, 051112 �2010�

1539-3755/2010/81�5�/051112�10� ©2010 The American Physical Society051112-1

�45� proposed a scheme exploiting trajectories in symbolicdynamics and measure of their complexity by running cylin-der or word entropies. From time averages on such entropies,an estimator is suggested in �45� for the signal-to-noise ratio.The estimator based on the Shannon entropy is proved in�45� to converge, for a sinusoidal signal in noise, asymptoti-cally at large signal-to-noise ratio. An improvement factor isdefined by the derivative of the estimator with respect to thesignal-to-noise ratio and �45� finds that there is a maximumof the improvement factor at intermediate noise levels, whichis interpreted as a form of stochastic resonance. When thescheme is used with the Rényi entropy, Ref. �45� reports ashift of the maximum of the improvement factor towardlower noise amplitudes. By contrast, our present study doesnot deal with the problem of estimating the signal-to-noiseratio of a sinusoid in noise, but it realizes the analysis of aninformation channel.

In the present paper, we will first review definitions andbasic properties of Rényi entropy based information mea-sures. We will especially present a practical problem, insource coding, where a specific operational role is assignedto the Rényi entropy in a configuration differing from thetraditional Shannon entropy. Next, we will present a binaryinformation channel and its analysis based on the Rényi en-tropy. We will demonstrate stochastic resonance or noise-aided transmission in this information channel and optimalityconditions that single out nontrivial orders for the Rényi en-tropy also differing from the traditional Shannon entropy. Aconfrontation of the quantitative information measures withvisual perception will also be proposed in an experiment ofnoise-aided binary image transmission. The results of thepaper seek to contribute in two directions: to enlarge thevision of the capabilities of Rényi-entropy-based informationmeasures and to consolidate stochastic resonance as a uni-versal phenomenon characterizable with general informationmeasures.

II. RÉNYI ENTROPY MEASURES AND PROPERTIES

A. Rényi entropy

From an alphabet of N symbols, an information sourceemits symbols independently with probabilities Pi, for i=1 toN. The Rényi entropy of the source is defined as �3�

H��Pi� =1

1 − �log��

i=1

N

Pi�� , �1�

for an order ��0. At the limit �=1, the L’Hospital ruleyields H1�Pi�=−�i=1

N Pi log�Pi�, i.e., the Shannon entropy.For any order ��0, the Rényi entropy H��Pi� of Eq. �1� isnonnegative and it reaches its maximum Hmax=log�N� atequiprobability Pi=1 /N for all i=1 to N �46�. H��Pi� is con-cave �� � for 0���1 and pseudoconcave �a single maxi-mum� for 1�� �47�. For a given probability distribution,H��Pi� is a decreasing function of � �46�, decaying fromH0�Pi�=log�N�=Hmax down to H��Pi�=−log�maxi�Pi��. TheRényi entropy is additive for independent random variables.

For illustration, Fig. 1 shows the Rényi entropy H��Pi� ofEq. �1� for a binary source �of interest to us in the sequel�.

Especially, Fig. 1 depicts how the entropy H��Pi� loses itsconcavity �� � at ��1 while keeping a single maximum�pseudoconcave�.

B. Rényi relative entropy

Associated with the Rényi entropy is the Rényi relativeentropy or divergence, which refers to two probability distri-butions �Pi and �Qi, i=1 to N, over the same alphabet, andis defined as �3,46�

D��Pi Qi� =1

� − 1log��

i=1

N

Pi�Qi

1−�� . �2�

At the limit �=1, the L’Hospital rule yields D1�Pi Qi�=�i=1

N Pi log�Pi /Qi�, i.e., the Kullback-Leibler relative en-tropy �2�. For any order ��0, the Rényi relative entropyD��Pi Qi� of Eq. �2� is nonnegative and vanishes if and onlyif Pi=Qi for all i=1 to N �46�.

For two given probability distributions �Pi and �Qi,i=1 to N, the Rényi relative entropy D��Pi Qi� is an increas-ing function of �. At the limit 0←�, one has D0←��Pi Qi�=�D1�Qi Pi�, meaning that the Rényi relative entropyD��Pi Qi� reaches zero as a lower bound when 0←�, withthe limiting behavior controlled by the Kullback-Leiblerrelative entropy D1�Qi Pi�. This is in accordance with a gen-eral property of D��Pi Qi� of Eq. �2� which for any 0���1 verifies �1−��D��Pi Qi�=�D1−��Qi Pi�. At the limit�→�, one has D�→��Pi Qi�=log�maxi�Pi /Qi�� as the upperbound reached by D��Pi Qi�. By choosing the referenceprobabilities �Qi as the uniform distribution �Qi=1 /N forall i=1 to N, one obtains

D��Pi Qi = 1/N� = Hmax − H��Pi� , �3�

expressing a connection between entropy and relative en-tropy at any Rényi order �.

C. Rényi transinformation

One now considers an input alphabet with N symbols, anoutput alphabet with M symbols, and over those two a joint

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

probability P1

Rén

yien

trop

y

α=0.4

α=20

1

FIG. 1. �Color online� Rényi entropy H��Pi� of Eq. �1�, as afunction of the probability P1 of a binary source �P1 ,1− P1, forthree values of the order �=0.4, �=20, and �=1 identified bycrosses ��� corresponding to the Shannon entropy.

CHAPEAU-BLONDEAU, ROUSSEAU, AND DELAHAIES PHYSICAL REVIEW E 81, 051112 �2010�

051112-2

probability distribution �Pij, for �i , j�� �1,N�� �1,M�, aswould occur between the emitting and receiving ends of acommunication channel. The N input symbols, indexed by i,have marginal probabilities Pi=� j=1

M Pij. The M output sym-bols, indexed by j, have marginal probabilities Qj =�i=1

N Pij. ARényi transinformation or mutual information follows as

I� = D��Pij PiQj� =1

� − 1log��

i=1

N

�j=1

M

Pij�Pi

1−�Qj1−�� .

�4�

In the case �=1 the Rényi transinformation I�=1 from Eq.�4� is the Shannon transinformation. The limit behaviors ofthe Rényi relative entropy D� indicated in Sec. II B give forthe Rényi transinformation I0←�=�D1�PiQj Pij� and I�→�

=log�maxi,j�Pij / PiQj��.

D. Source coding with the Rényi entropy

We now describe a practical problem of source codingintroduced in �48� and in the resolution of which the Rényientropy emerges at an order � differing from the traditionalShannon entropy. The N symbols of a source alphabet arecoded with an encoding alphabet of D characters. Symbol i,having probability Pi, is coded by a word with a length �i ofD-ary characters, for i=1 to N. For a uniquely decipherablecode, the lengths �i must satisfy �2,49� the Kraft inequality

�i=1

N

D−�i � 1. �5�

The traditional approach to optimal source coding �2,49�is to measure the elementary cost ci of encoding symbol idirectly by its code length �i=ci and then to seek thoselengths �i that minimize the average coding length �i=1

N Pi�iwhile satisfying the constraint �5�. The optimal lengths comeout as �i=−logD�Pi�, for all i=1 to N, and these achieve theminimum average coding length −�i=1

N Pi logD�Pi� which isthe Shannon entropy of the source. For all other code lengths�i, the Shannon entropy forms a lower bound to the averagecoding length �i=1

N Pi�i.A less traditional approach to optimal source coding �48�

is to measure the elementary cost of encoding symbol i asci=D�i, introducing a cost ci which is an exponential func-tion of the code length �i, with a parameter �0 to have thecost ci an increasing function of the length �i. The global costof encoding the source is expressed by the average

C = �i=1

N

Pici = �i=1

N

PiD�i. �6�

Minimizing the average cost C of Eq. �6� is equivalent tominimizing the monotonic increasing function of C as

L =1

logD��

i=1

N

PiD�i� . �7�

By measuring the coding performance with L of Eq. �7�, thetraditional approach can be recovered as a special case. At

the limit 0← in Eq. �7�, the L’Hospital rule yields L0=�i=1

N Pi�i which is the traditional average coding length.Also, at the limit →�, Eq. �7� yields L�=maxi��i� puttingall the weight on the longest code word. For intermediatevalues of , the quantity L of Eq. �7� is interpretable as ageneralized average coding length1 of order and is a non-decreasing function of verifying L0�L�L�. As 0←,the average coding length L tends to distribute the weightsamong the code words in proportion of their lengths, whileas →� more weight is put on the long code words.

Now this extended approach to optimal source coding�48� seeks those lengths �i that minimize the generalizedaverage coding length L of Eq. �7� while satisfying the con-straint �5�. The optimal lengths come out as

�i = − logD

Pi�

�j=1

N

Pj�

= − � logD�Pi� + �1 − ��H��Pi� , �8�

for all i=1 to N, with �=1 / �+1�. And the optimal lengths�i of Eq. �8� achieve in Eq. �7� the minimum average codinglength H��Pi� which is the Rényi entropy of order � of thesource. For all other code lengths �i, the Rényi entropyH��Pi� at �=1 / �+1� forms a lower bound to the general-ized average coding length L of Eq. �7�. These results con-tain, as a special case, the traditional approach to optimalsource coding, in the limit 0←. This generalized approachto optimal coding finds application to minimize the probabil-ity of buffer overflow �51� or other design optimization incommunication systems �52,53�. These results are further de-veloped in �46,50,54�. They are especially interesting sincearguably they represent, among the rare instances of thiskind, the most simple and concrete, yet fundamental, situa-tion where a special and operational role is assigned to theRényi entropy at an order � differing from the traditionalShannon case �=1. This is realized at the occasion of asource coding problem where the generalized �nonlinear� av-erage coding length being optimized determines the non-trivial order ��1 of the Rényi entropy. In the sequel, wewill present another process, under the form of a noise-aidedinformation transmission over a binary channel, which alsopoints to a nontrivial order ��1 for the Rényi entropy.

III. BINARY INFORMATION CHANNEL

An information channel emits input symbols X from thebinary alphabet �0,1. The successive input symbols are in-dependent and identically distributed with the probabilitiesP1=Prob�X=1 and P0=1− P1=Prob�X=0. At the receivingend of the channel, the output symbols Y are in the binaryalphabet �0,1. Transmission over the channel is character-

1Two interpretations are possible for Eqs. �6� and �7�: an exponen-tial elementary cost ci=D�i associated with a conventional linearaverage in Eq. �6� �followed by the increasing transformation of Eq.�7� with no change to the minimizer� or a conventional linear el-ementary cost ci=�i associated with a generalized �50� exponentialaverage −1��iPi��i�� realized by Eq. �7� with the strictly increas-ing function �u�=Du.

RÉNYI ENTROPY MEASURE OF NOISE-AIDED … PHYSICAL REVIEW E 81, 051112 �2010�

051112-3

ized by the four conditional probabilities Pj�i=Prob�Y = j �X= i, for �i , j�� �0,12. The joint input-output probabilitiesresult as Pij = Pj�iPi and the output probabilities Qj =Prob�Y= j=�i=0

1 Pj�iPi. Thus in Eq. �4�, one has Pij�Pi

1−�= Pj�i� Pi and

the input-output Rényi transinformation follows as

I��X;Y� =1

� − 1log��

j=0

1

�Pj�0� P0 + Pj�1

� P1�Qj1−�� . �9�

Equation �9� is the input-output Rényi transinformationfor any �memoryless� binary channel characterized by thefour transmission probabilities Pj�i. We now specify concretephysical conditions that determine a definite channel and itsprobabilities Pj�i. We consider the binary input X in the trans-mission corrupted by a white noise W to yield X+W and thenat the receiver X+W is compared to a fixed decoding thresh-old � to determine the binary output Y of the channel accord-ing to

If X + W � � then Y = 1,

else Y = 0.�10�

The noise W has the cumulative distribution function F�w�=Prob�W�w. The input X and the noise W are statisticallyindependent.

The input-output transmission probabilities of this binarychannel are readily derived. For instance, the probabilityP0�1=Prob�Y =0 �X=1 is also Prob�X+W�� �X=1 whichamounts to Prob�W��−1=F��−1�. With similar rules, onearrives at

P0�1 = Prob�Y = 0�X = 1 = F�� − 1� , �11�

P1�1 = Prob�Y = 1�X = 1 = 1 − F�� − 1� , �12�

P0�0 = Prob�Y = 0�X = 0 = F��� , �13�

P1�0 = Prob�Y = 1�X = 0 = 1 − F��� . �14�

These transmission probabilities Pj�i, �i , j�� �0,12, define anasymmetric binary channel. For this channel, a typical evo-lution of the Rényi transinformation I��X ;Y� of Eq. �9� isshown in Fig. 2, with the binary input X=0 or 1 evolving onboth sides of the decoding threshold �=0.8. In such condi-tion, the presence of the channel noise W in Eq. �10� hindersthe recovery of the information signal at the receiving end. Itresults that the performance of the transmission, as measuredby the input-output Rényi transinformation I��X ;Y�, de-creases as the level of noise increases, as visible in Fig. 2.

In Fig. 2, a similar decreasing evolution of the Rényitransinformation I��X ;Y� as the level of noise increases isobserved for any order �, especially, but not only, in theShannon case �=1. In this respect, this shows that the Rényitransinformation I��X ;Y� at any order � is capable of mani-festing the detrimental action of the noise in the transmissionof information through the channel. We will now consideranother regime of operation of the channel of Eq. �10� andshow the possibility of a constructive action of the noise inthe transmission of information assessed with the Rényitransinformation I��X ;Y�.

IV. NOISE-IMPROVED INFORMATION TRANSMISSION

A. Noise-improved Rényi transinformation

For an information channel with a decoding threshold �=1.2 in Eq. �10�, the evolution of the input-output Rényitransinformation I��X ;Y� from Eq. �9� is presented in Fig. 3.The results of Fig. 3 clearly demonstrate a nonmonotonicaction of the noise. In the conditions of Fig. 3, the binaryinput X by itself is always below the response threshold �=1.2 on the output. As a consequence, in the absence ofnoise at �=0 in Fig. 3, the channel output permanently re-mains at Y =0. No information is transmitted through thechannel, as expressed by the Rényi transinformation I��X ;Y�which stays at zero when �=0 for any finite order �. How-ever, as the noise level � is progressively raised above zeroin Fig. 3, a cooperative effect can take place, with the noiseW assisting the subthreshold input X to overcome the re-sponse threshold �. This elicits transitions in the output Ybearing statistical dependence with the input X. As a conse-quence, a nonzero input-output transmission of informationoccurs, as registered by the Rényi transinformation I��X ;Y�which starts to increase in Fig. 3 as the noise level � risesabove zero. There exists a nonzero amount of noise forwhich the information transfer measured by I��X ;Y� is maxi-mized and such a maximum of I��X ;Y� occurs for any finiteorder � as shown in Fig. 3. This is the effect of stochasticresonance or noise-aided information transmission, regis-tered by the Rényi transinformation I��X ;Y� in Fig. 3 at anyfinite order �.

In Fig. 3, the order �=1 corresponds to the situationwhere I�=1�X ;Y� is the Shannon transinformation andI�=1�X ;Y� in Fig. 3 culminates at a maximum for a nonzerolevel of noise �, as also reported in �42�. For the limit order0←�, the Rényi transinformation goes to zero as

I0←��X;Y� = �D1�PiQj Pij� , �15�

and meanwhile, I0←��X ;Y� of Eq. �15� keeps a maximum asa function of � which is a maximum inherited from

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

noise rms amplitude

Rén

yitr

ansi

nfor

mat

ion

α=0.2

α=2

FIG. 2. �Color online� Input-output Rényi transinformationI��X ;Y� from Eq. �9�, as a function of the rms amplitude � of thezero-mean Gaussian noise W, for an information channel with inputprobability P1=0.45 and threshold �=0.8. The order � goes from0.2 to 2 with step 0.2. The crosses ��� identify �=1 whenI�=1�X ;Y� is the Shannon transinformation.

CHAPEAU-BLONDEAU, ROUSSEAU, AND DELAHAIES PHYSICAL REVIEW E 81, 051112 �2010�

051112-4

D1�PiQj Pij�. It is the location of this maximum, occurringalso at a nonzero level of noise �, which is indicated by acircle on the curve of I�=0�X ;Y� in Fig. 3.

For the limit order �→�, the Rényi transinformation be-haves as I�→��X ;Y�=log�maxi,j�Pj�i /Qj��. For the binarychannel resulting from Eq. �10�, we have P0�0� P0�1 andP1�1� P1�0, so that

maxi,j

�Pj�i/Qj� = max�P0�0

Q0,P1�1

Q1� . �16�

And from Eqs. �11�–�14�, it comes

maxi,j

�Pj�i/Qj� = max� F���P0F��� + P1F�� − 1�

,1 − F�� − 1�

P0�1 − F���� + P1�1 − F�� − 1��� . �17�

In this way, Eq. �17� provides access to the Rényi transinfor-mation I�→��X ;Y�=log�maxi,j�Pj�i /Qj�� especially as a func-tion of the noise level � intervening through the cumulativedistribution function F� · � of the channel noise W. In Eq.�17�, the maximum in the right-hand side switches from oneterm to the other depending on � �at given P1 and ��. Anevolution resulting for I�→��X ;Y� is depicted in Fig. 3�b�. Inthe conditions of Fig. 3, at P1=0.45 and ��1, the maximumin Eq. �17� is realized by the second term in the right-handside to yield

I�→��X;Y� = log� 1 − F�� − 1�P0�1 − F���� + P1�1 − F�� − 1��� ,

�18�

which forms a decreasing function of �, depicted in Fig.3�b�. The maximum of I�→��X ;Y� occurs at �=0 and comesout as −log�P1� indicated in Fig. 3�b� as −log2�0.45�=1.152.The order �→� is the only configuration where the maxi-mum of I��X ;Y� occurs at �=0, while for any finite order �,the maximum of I��X ;Y� occurs at a nonzero level � of thechannel noise. This is the manifestation of the stochastic

resonance effect, taking place with any finite order � of theRényi transinformation I��X ;Y�.

It is visible in Fig. 3 that the maximum of the Rényitransinformation I��X ;Y� occurs at an optimal level �opt ofthe noise which varies with the Rényi order �. This variationof �opt with � is further analyzed in Fig. 4 for differentvalues of the decoding threshold �.

A remarkable property observed in Fig. 4 is that the opti-mal noise level �opt maximizing I��X ;Y� experiences a non-monotonic evolution with the order �. There exists in Fig. 4an optimal value �opt=2.20 of the Rényi order where �opt ismaximized. At �opt=2.20 in Fig. 4, the maximum reached by�opt is usually dependent on the decoding threshold � and�opt increases as � increases since more noise is required toassist the subthreshold input at higher threshold. Neverthe-less, the nonmonotonic evolution of �opt maximized at �optin Fig. 4 demonstrates that the stochastic resonance effectselects a specific order �opt of the Rényi transinformationI��X ;Y�. This optimal order �opt identifies the Rényi transin-formation I�opt

�X ;Y� that is capable of drawing the most pro-nounced benefit of the added noise in stochastic resonance,since I�opt

�X ;Y� stands as the measure of input-output infor-

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

noise rms amplitude

Rén

yitr

ansi

nfor

mat

ion

α=0

α=5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

noise rms amplitude

Rén

yitr

ansi

nfor

mat

ion

α=0

α=10

α=∞

(b)(a)

FIG. 3. �Color online� Input-output Rényi transinformation I��X ;Y� from Eq. �9�, as a function of the rms amplitude � of the zero-meanGaussian noise W, for an information channel with input probability P1=0.45 and threshold �=1.2. On each curve, the maximum is indicatedby a circle ���, except for �=1 identified by a cross ��� when I�=1�X ;Y� is the Shannon transinformation. The order � goes from 0 to 5 withstep 0.1 �panel A� and from 0 to 10 with step 0.5, then �=20, 30, 40, 50, � �panel B�.

RÉNYI ENTROPY MEASURE OF NOISE-AIDED … PHYSICAL REVIEW E 81, 051112 �2010�

051112-5

mation transfer that gets maximized at the largest optimalnoise level �opt.

Additionally, another property also observed in Fig. 4 is aquasi-invariance of the optimal Rényi order �opt with thedecoding threshold �. Due to the intricacy of the �nonlinear�dependencies involved in the Rényi transinformationI��X ;Y�, we were not able to obtain an analytical character-ization of the theoretical relation implied between �opt and �.Alternatively, we have performed a numerical characteriza-tion which presents in Fig. 5 the optimal Rényi order �opt, asa function of the input probability P1, at different values ofthe decoding threshold �. The results of Fig. 5 show that inthe region around P1�0.5, which corresponds to an approxi-mately balanced binary input, almost no influence of the de-coding threshold is observed on the optimal order �opt. Insuch conditions, the optimal order �opt appears intrinsic tothe channel structure and independent of the decodingthreshold �, at least in the range tested in Fig. 5 for �. Theseare the conditions that prevail in Fig. 4, at P1=0.45, with nodetected dependence of �opt on �. For small or large values

of P1, far from the balanced case P1�0.5, a dependence of�opt on � gradually appears, as visible in Fig. 5. Neverthe-less, this dependence, whenever it applies, does not criticallyimpact the essential property of the existence of an optimalorder �opt selected by stochastic resonance in the operationof the binary channel.

It is also interesting to note that this optimal Rényi order�opt selected by stochastic resonance at its maximum usuallydiffers from the Shannon order �=1. The Rényi transinfor-mation I��X ;Y� that best exploits the stochastic resonance isusually not the Shannon transinformation I1�X ;Y�. It is nei-ther a degenerate configuration I�=0�X ;Y� nor I�=��X ;Y� oc-curring as the limit of a monotonic evolution. It is a trulynontrivial instance I�opt

�X ;Y� that usually emerges for theRényi transinformation in the presence of stochastic reso-nance.

B. Image transmission

As another point of view complementing the quantifica-tion of stochastic resonance performed in Sec. IV A with theRényi transinformation, a visual illustration of the noise-enhanced information transmission is proposed in an experi-ment with images. The early study of �55� reported stochasticresonance or improvement by noise in an experiment of vi-sual perception of images, with an evaluation based on thepsychovisual assessment performed by human subjects. Weextend this approach with a similar experiment of imagetransmission, associated with an evaluation by quantitativeinformation measures. We consider an experiment of binaryimage transmission. The binary input X is a black and whiteimage where the probability P1=Prob�X=1 is the fractionof white pixels. A noise W is added, followed by the thresh-old decoding according to Eq. �10� to reconstruct a black andwhite output image Y. Various output images Y are shown inFig. 6 at different levels of the noise W on the transmissionchannel, in conditions matching those quantified in Figs. 3and 4.

The nonmonotonic action of the noise is visually perceiv-able on the images of Fig. 6. For a subthreshold input imageX in Fig. 6, when the level � of the noise is too small as inFig. 6�a�, a poor reconstructed image Y is obtained at theoutput. For intermediate noise levels as in Figs. 6�b� and

0 1 2 3 4 5 6 7 8 9 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

Rényi order α

optim

alno

ise

rms

ampl

itude

θ=1.1

θ=1.2

θ=1.5

θ=2

FIG. 4. Optimal noise rms amplitude �opt maximizing the input-output Rényi transinformation I��X ;Y� from Eq. �9�, as a functionof the order �. The information channel is with input probabilityP1=0.45, zero-mean Gaussian noise W, and threshold �.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.80

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

input probability

optim

alR

ényi

orde

r

FIG. 5. �Color online� Optimal Rényi order �opt, as a function ofthe input probability P1, for the binary channel of Eq. �10� withthreshold �=1.1 ���, 1.2�+�, 1.5�� �, and 2 ���.

(a) (b) (c) (d)

FIG. 6. Binary image Y at the output of an information channelaccording to Eq. �10�. Black and white input image X has size610�555 pixels, with the fraction P1=0.45 of white pixels. Thedecoding threshold �=1.2. The channel noise W is zero-meanGaussian with rms amplitude �a� �=0.1, �b� �=0.46 maximizingthe Shannon transinformation I1�X ;Y� in Fig. 3, �c� �=0.52 maxi-mizing the Rényi transinformation I�opt

�X ;Y� at the optimal order�opt=2.20 in Fig. 3, and �d� �=1.

CHAPEAU-BLONDEAU, ROUSSEAU, AND DELAHAIES PHYSICAL REVIEW E 81, 051112 �2010�

051112-6

6�c�, there is a favorable action of the noise which assists thesubthreshold input image to overcome the decoding thresh-old. This results in a much better quality of image recon-struction in Figs. 6�b� and 6�c� at the output, thanks to theaction of the noise. Further, at still higher noise level as inFig. 6�d�, the noise gradually recovers its detrimental impactresulting in poorer quality of the reconstructed image Y atthe output.

When improvement by noise of image transmission takesplace in Figs. 6�b� and 6�c�, the noise level is set succes-sively at the maximum of the Shannon transinformationI1�X ;Y� and of the Rényi transinformation I�opt

�X ;Y� at theoptimal order �opt=2.20, as deduced from Figs. 3 and 4. Thevisual perception in Figs. 6�b� and 6�c� also records im-proved image transmission in these two configurations. Thisexpresses that I1�X ;Y� or I�opt

�X ;Y� are both acceptable mea-sures to quantify the benefit from noise in image transmis-sion as perceived visually and that also no one measureemerges against the other as being more suitable for thispurpose. For noise-improved image transmission, other per-formance measures more specific to images have also beenshown appropriate for quantifying stochastic resonance �56�.In this respect, the main point of the present study is todemonstrate the ability of a general information measuresuch as the Rényi transinformation to quantify stochasticresonance. And the present experiment with images showsthat the optimal Rényi measure I�opt

�X ;Y� emerging with sto-chastic resonance is consistent with noise-improved imagetransmission as registered by visual perception.

C. Noise-improved Rényi information capacity

The optimal Rényi order �opt selected by the stochasticresonance as discussed in Sec. IV A is usually related to agiven information source characterized by the input probabil-ity P1. This is for instance illustrated by the evolutions of�opt with P1 in Fig. 5. A point of view not impacted by thisdependence with P1 is accessible by considering the Rényiinformation capacity of the channel. The Rényi transinforma-tion I��X ;Y� observed in Figs. 3 and 4 assesses the transmis-sion through the noisy channel of a binary input X with givenprobability P1=Prob�X=1=1−Prob�X=0. The Rényi in-formation capacity C� is defined by seeking the value P1

� ofP1 achieving the maximum C� of I��X ;Y� at given order �.For the optimal input probability P1

�, an explicit analyticalexpression exists in the special case �=1 of the Shannoninformation capacity C�=1 of the asymmetric binary channel,taking the form �42�

P1� =

AP0�0 − 1

A�P0�0 − P0�1�, �19�

with

A = 1 + exp h�P0�0� + h�P1�0� − h�P1�1� − h�P0�1�

P0�0 − P0�1� ,

�20�

with the function h�u�=−u ln�u�. Then, the Shannon capacityC�=1 follows from Eq. �19� in I��X ;Y� at �=1. For an arbi-

trary order � other than �=1, the optimal probability P1�

usually needs to be computed numerically by maximizingI��X ;Y� from Eq. �9�. In this way, we have realized an evalu-ation of the optimal probability P1

� in Fig. 7, and of thecorresponding Rényi information capacity C� in Fig. 8, forthe binary channel according to Eq. �10�.

In addition to the case �=1 ruled by Eq. �19�, analyticalinsight on the capacity C� can also be obtained in two otherlimit cases for the Rényi order �. In the limit 0←�, theRényi transinformation I0←��X ;Y� of Eq. �15� is ruled byD1�PiQj Pij� which takes its maximum at P1

��1 /2, espe-cially P1

�→1 /2 when the noise level � goes to zero or toinfinity. In these conditions, the capacity C0←� as well as theRényi transinformation I0←��X ;Y� of Eq. �15� go to zero.The evolutions to these limit behaviors for P1

� and C0←� are

0 0.5 1 1.5 20

0.1

0.2

0.3

0.4

0.5

0.6

α=0.2

α=4

noise rms amplitude

optim

alin

putp

roba

bilit

y

FIG. 7. �Color online� Optimal input probability P1� achieving

the Rényi information capacity C�, as a function of the rms ampli-tude � of the zero-mean Gaussian noise W, for an information chan-nel with decoding threshold �=1.2. The order � goes from 0.2 to 4with step 0.2. Crosses ��� identify �=1 when P1

� achieves theShannon information capacity C�=1 according to Eq. �19�.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

noise rms amplitude

Rén

yiin

form

atio

nca

paci

ty

α=0

α=2.3

FIG. 8. �Color online� Input-output Rényi information capacityC�, as a function of the rms amplitude � of the zero-mean Gaussiannoise W, for an information channel with decoding threshold �=1.2. On each curve, the maximum is indicated by a circle ���,except for �=1 identified by a cross ��� when C�=1 is the Shannoninformation capacity. The order � goes from 0 to 2.3 with step 0.1.

RÉNYI ENTROPY MEASURE OF NOISE-AIDED … PHYSICAL REVIEW E 81, 051112 �2010�

051112-7

discernable in Figs. 7 and 8, respectively. At the limit �→�, the Rényi transinformation I�→��X ;Y� from Eq. �17� ismaximized by P1

�→0 to yield the capacity C�→�=log��1−F��−1�� / �1−F���� which is the limit of Eq. �18� asP1

�→0.Compared to the Rényi transinformation I��X ;Y� at fixed

P1 as in Fig. 3, the same remarkable properties related tostochastic resonance are observed for the Rényi informationcapacity C� at the optimal P1

� in Fig. 8. At any order �, theRényi information capacity C� undergoes a nonmonotonicevolution as the noise level � increases. In the regime ��1 of a subthreshold binary input X, when no noise ispresent, no information is transmitted, as marked by a van-ishing Rényi capacity C� at any order � when �=0 in Fig. 8.Adding noise then modifies the channel, in a way where thesubthreshold input X=1 has more chance to get across andbe correctly decoded as Y =1 at the output. This constructiveaction of the noise induces a capacity C� rising above zero inFig. 8. Moreover, a nonzero level of noise exists where thecapacity C� is maximized in Fig. 8. This is again a manifes-tation of the stochastic resonance with a Rényi informationcapacity C� maximized at a nonzero optimal level of noiseon the channel. Also, as in Fig. 3, the optimal noise level �optmaximizing C� in Fig. 8 is found dependent on the Rényiorder �, yet with a nonmonotonic dependence. This depen-dence of �opt with � is represented in Fig. 9.

The nonmonotonic dependence in Fig. 9 identifies an op-timal Rényi order �opt=1.44 at which the optimal noise level�opt of stochastic resonance assumes its largest value. Andthis optimal order �opt=1.44 is found in Fig. 9 invariant withthe decoding threshold � of the channel. This invariance is atleast observed, as in Fig. 4, in the absence of a theoreticalproof, at the precision of our numerical analysis and for the“reasonable” range of � tested in Fig. 9. However, indepen-dently of this invariance with the channel threshold �, wehave now a characterization of an optimal Rényi order �optwhich is not attached to a given information source via itsP1. The stochastic resonance selects a nontrivial Rényi order�opt=1.44 through the Rényi information capacity C�opt

thatexploits stochastic resonance in the most pronounced waysince C�opt

gets maximized by the largest possible optimal

noise level �opt. The optimal Rényi order �opt=1.44 selectedby stochastic resonance in the capacity is now intrinsic to thebinary channel and insensitive to the input probability �andto the decoding threshold�. And the optimal order �opt, se-lected by stochastic resonance in the Rényi information ca-pacity, differs from the Shannon order �=1.

V. DISCUSSION AND CONCLUSION

After reviewing basic properties of generalized informa-tion measures based on the Rényi entropy, we have appliedthese generalized measures for the analysis of a binary infor-mation channel. This analysis contains as a special case: theclassic reference model of information transmission over abinary channel quantified with Shannon entropy based mea-sure. The analysis with the Rényi entropy therefore offers anextended reference model useful to describe informationtransmission in broader conditions. We have exploited thisextended model to investigate further possibilities and prop-erties of stochastic resonance or noise-aided informationtransmission. The results demonstrate that stochastic reso-nance occurs and is registered by the Rényi entropy mea-sures at any finite order �, including the Shannon order �=1. Moreover, in definite conditions when one seeks the Ré-nyi information measure that best exploits stochastic reso-nance, i.e., the information measure that is maximized by thelargest optimal amount of noise, then a nontrivial order �optdiffering from the Shannon case �=1 usually emerges. Inthis way, in binary information transmission over the chan-nel, stochastic resonance selects a specific nontrivial Rényimeasure of information differing from the classic Shannonmeasure.

The optimal Rényi information measure can be selectedas the Rényi transinformation I�opt

�X ;Y� associated with agiven information source to be transmitted. In this case, theoptimal order �opt, such as the Rényi transinformationI��X ;Y� itself, is usually dependent on the input probabilityP1 of the binary information source applied to the channel.The optimal Rényi transinformation I�opt

�X ;Y� is in this wayoptimal for transmission of a given information source. Sucha dependence of the optimal Rényi order with the binaryinformation source disappears when one turns to the Rényicapacity C� of the channel. We have shown that the capacityC�, as well as I��X ;Y�, is capable of registering a noise-aided information transmission at any finite order �. And ingeneral, the optimal order selected for I�opt

�X ;Y� or C� bystochastic resonance differs from the Shannon order �=1.

In addition to the demonstration of feasibility of stochas-tic resonance in the binary channel, it therefore appears as aremarkable and robust observation that stochastic resonanceacts to select an optimal Rényi information measure, underthe form of an optimal Rényi transinformation I�opt

�X ;Y� fora given information source, or of an optimal Rényi capacityC�opt

. Beyond its nominal defining condition of optimality, itremains difficult for the moment to further specify a moreconcrete interpretation to the optimal Rényi order �optemerging in different situations of stochastic resonance. Onthis issue, we have conducted an experiment of noise-aidedbinary image transmission. It showed that the assessment of

0 0.5 1 1.5 2 2.50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Rényi order α

optim

alno

ise

rms

ampl

itude

θ=1.1

θ=1.2

θ=1.5

θ=2

FIG. 9. Optimal noise rms amplitude �opt maximizing the Rényiinformation capacity C� as a function of the order �. The informa-tion channel is with zero-mean Gaussian noise W and threshold �.

CHAPEAU-BLONDEAU, ROUSSEAU, AND DELAHAIES PHYSICAL REVIEW E 81, 051112 �2010�

051112-8

the improvement by the optimal Rényi measure is consistentwith the visual perception, as the Shannon transinformationalso is. In any case, we find it helpful to have access todefinite quantitative processes capable of assigning specialroles or properties to Rényi information measures at non-trivial orders differing from the Shannon order. This mayserve for getting more insight on the various possible ways

of quantitatively measuring information, in different contextsand for different prospects where information contents arerelevant and need be formalized. In this way, the results ofthe paper want to contribute in two directions: to consolidatestochastic resonance as a universal phenomenon characteriz-able with general information measures and to enlarge theappreciation of quantitative measures of information.

�1� C. E. Shannon, Bell Syst. Tech. J. 27, 379 �1948�.�2� T. M. Cover and J. A. Thomas, Elements of Information

Theory �Wiley, New York, 1991�.�3� A. Rényi, Proceedings 4th Berkeley Symposium on Mathemati-

cal Statistics and Probability �University of California Press,Berkeley, 1961�, pp. 547–561.

�4� L. Gammaitoni, P. Hänggi, P. Jung, and F. Marchesoni, Rev.Mod. Phys. 70, 223 �1998�.

�5� F. Chapeau-Blondeau and D. Rousseau, Fluct. Noise Lett. 2,L221 �2002�.

�6� M. D. McDonnell, N. G. Stocks, C. E. M. Pearce, and D.Abbott, Stochastic Resonance: From Suprathreshold Stochas-tic Resonance to Stochastic Signal Quantization �CambridgeUniversity Press, Cambridge, England, 2008�.

�7� M. Löcher, D. Cigna, and E. R. Hunt, Phys. Rev. Lett. 80,5212 �1998�.

�8� D. G. Luchinsky, R. Mannella, P. V. E. McClintock, and N. G.Stocks, IEEE Trans. Circuits Syst., II: Analog Digital SignalProcess. 46, 1205 �1999�.

�9� D. Rousseau and F. Chapeau-Blondeau, Signal Process. 85,571 �2005�.

�10� A. Bulsara, A. J. Maren, and G. Schmera, Biol. Cybern. 70,145 �1993�.

�11� B. J. Gluckman, T. I. Netoff, E. J. Neel, W. L. Ditto, M. L.Spano, and S. J. Schiff, Phys. Rev. Lett. 77, 4098 �1996�.

�12� X. Godivier and F. Chapeau-Blondeau, Europhys. Lett. 35,473 �1996�.

�13� W. C. Stacey and D. M. Durand, J. Neurophysiol. 83, 1394�2000�.

�14� B. Andò, S. Baglio, S. Graziani, and N. Pitrone, Int. J. Elec-tron. 86, 86 �1999�.

�15� L. Gammaitoni and A. R. Bulsara, Phys. Rev. Lett. 88, 230601�2002�.

�16� F. Chapeau-Blondeau and D. Rousseau, Phys. Rev. E 70,060101�R� �2004�.

�17� B. M. Jost and B. E. A. Saleh, Opt. Lett. 21, 287 �1996�.�18� M. Misono, T. Kohmoto, Y. Fukuda, and M. Kunitomo, Opt.

Commun. 152, 255 �1998�.�19� S. Blanchard, D. Rousseau, D. Gindre, and F. Chapeau-

Blondeau, Opt. Lett. 32, 1983 �2007�.�20� M. Ozer, M. Perc, and M. Uzuntarla, Phys. Lett. A 373, 964

�2009�.�21� M. Perc, Phys. Rev. E 76, 066203 �2007�.�22� L. Martínez, T. Pérez, C. R. Mirasso, and E. Manjarrez, J.

Neurophysiol. 97, 4007 �2007�.�23� E. Manjarrez, I. Mendez, L. Martinez, A. Flores, and C. R.

Mirasso, Neurosci. Lett. 415, 231 �2007�.�24� M. Perc and M. Marhl, Phys. Rev. E 71, 026229 �2005�.�25� D. C. Gong, G. Hu, X. D. Wen, C. Y. Yang, G. R. Qin, R. Li,

and D. F. Ding, Phys. Rev. A 46, 3243 �1992�.�26� F. Chapeau-Blondeau and D. Rousseau, Phys. Lett. A 351, 231

�2006�.�27� J. J. Collins, C. C. Chow, and T. T. Imhoff, Phys. Rev. E 52,

R3321 �1995�.�28� F. Chapeau-Blondeau and J. Rojas-Varela, Int. J. Bifurcat.

Chaos Appl. Sci. Eng. 10, 1951 �2000�.�29� M. D. McDonnell, D. Abbott, and C. E. M. Pearce, Fluct.

Noise Lett. 2, L205 �2002�.�30� M. E. Inchiosa and A. R. Bulsara, Phys. Rev. E 53, R2021

�1996�.�31� V. Galdi, V. Pierro, and I. M. Pinto, Phys. Rev. E 57, 6470

�1998�.�32� M. D. McDonnell, D. Abbott, and C. E. M. Pearce, Microelec-

tron. J. 33, 1079 �2002�.�33� F. Chapeau-Blondeau, Phys. Rev. E 66, 032101 �2002�.�34� H. Chen, P. K. Varshney, and J. H. Michels, IEEE Trans. Sig-

nal Process. 56, 5074 �2008�.�35� P. E. Greenwood, L. M. Ward, and W. Wefelmeyer, Phys. Rev.

E 60, 4687 �1999�.�36� Y. Wang and L. Wu, Fluct. Noise Lett. 5, L435 �2005�.�37� F. Chapeau-Blondeau, S. Blanchard, and D. Rousseau, Phys.

Rev. E 74, 031102 �2006�.�38� A. R. Bulsara and A. Zador, Phys. Rev. E 54, R2185 �1996�.�39� G. Deco and B. Schürmann, Physica D 117, 276 �1998�.�40� N. G. Stocks, Phys. Rev. E 63, 041114 �2001�.�41� D. Rousseau, J. Rojas Varela, and F. Chapeau-Blondeau, Phys.

Rev. E 67, 021102 �2003�.�42� F. Chapeau-Blondeau, Phys. Rev. E 55, 2016 �1997�.�43� X. Godivier and F. Chapeau-Blondeau, Int. J. Bifurcat. Chaos

Appl. Sci. Eng. 8, 581 �1998�.�44� L. B. Kish, G. P. Harmer, and D. Abbott, Fluct. Noise Lett. 1,

L13 �2001�.�45� P. beim Graben, Phys. Rev. E 64, 051104 �2001�.�46� I. Csiszar, IEEE Trans. Inf. Theory 41, 26 �1995�.�47� M. Ben-Bassat and J. Raviv, IEEE Trans. Inf. Theory 24, 324

�1978�.�48� L. L. Campbell, Inf. Control 8, 423 �1965�.�49� D. J. C. MacKay, Information Theory, Inference, and Learning

Algorithms �Cambridge University Press, Cambridge, England,2003�.

�50� M. B. Baer, IEEE Trans. Inf. Theory 52, 4380 �2006�.�51� P. A. Humblet, IEEE Trans. Inf. Theory 27, 230 �1981�.

RÉNYI ENTROPY MEASURE OF NOISE-AIDED … PHYSICAL REVIEW E 81, 051112 �2010�

051112-9

�52� A. C. Blumer and R. J. McEliece, IEEE Trans. Inf. Theory 34,1242 �1988�.

�53� M. B. Baer, IEEE Trans. Inf. Theory 54, 1273 �2008�.�54� J. F. Bercher, Phys. Lett. A 373, 3235 �2009�.

�55� E. Simonotto, M. Riani, C. Seife, M. Roberts, J. Twitty, and F.Moss, Phys. Rev. Lett. 78, 1186 �1997�.

�56� D. Rousseau, A. Delahaies, and F. Chapeau-Blondeau, IEEESignal Process. Lett. 17, 36 �2010�.

CHAPEAU-BLONDEAU, ROUSSEAU, AND DELAHAIES PHYSICAL REVIEW E 81, 051112 �2010�

051112-10

References with the titles of the cited papers:

[1] C. E. Shannon, “A mathematical theory of communication,” Bell System Technical Journal, vol. 27, pp. 379–423, 623–656,1948.

[2] T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.[3] A. Renyi, “On measures of entropy and information,” in Proceedings 4th Berkeley Symposium on Mathematical Statistics

and Probability, (Berkeley, USA), pp. 547–561, University of California Press, 1961.[4] L. Gammaitoni, P. Hanggi, P. Jung, and F. Marchesoni, “Stochastic resonance,” Reviews of Modern Physics, vol. 70,

pp. 223–287, 1998.[5] F. Chapeau-Blondeau and D. Rousseau, “Noise improvements in stochastic resonance: From signal amplification to optimal

detection,” Fluctuation and Noise Letters, vol. 2, pp. L221–L233, 2002.[6] M. D. McDonnell, N. G. Stocks, C. E. M. Pearce, and D. Abbott, Stochastic Resonance: From Suprathreshold Stochastic

Resonance to Stochastic Signal Quantization. Cambridge: Cambridge University Press, 2008.[7] M. Locher, D. Cigna, and E. R. Hunt, “Noise sustained propagation of a signal in coupled bistable electronic elements,”

Physical Review Letters, vol. 80, pp. 5212–5215, 1998.[8] D. G. Luchinsky, R. Mannella, P. V. E. McClintock, and N. G. Stocks, “Stochastic resonance in electrical circuits – I:

Conventional stochastic resonance,” IEEE Transactions on Circuits and Systems – II: Analog and Digital Signal Processing,vol. 46, pp. 1205–1214, 1999.

[9] D. Rousseau and F. Chapeau-Blondeau, “Constructive role of noise in signal detection from parallel arrays of quantizers,”Signal Processing, vol. 85, pp. 571–580, 2005.

[10] A. Bulsara, A. J. Maren, and G. Schmera, “Single effective neuron: Dendritic coupling effects and stochastic resonance,”Biological Cybernetics, vol. 70, pp. 145–156, 1993.

[11] B. J. Gluckman, T. I. Netoff, E. J. Neel, W. L. Ditto, M. L. Spano, and S. J. Schiff, “Stochastic resonance in a neuronalnetwork from mammalian brain,” Physical Review Letters, vol. 77, pp. 4098–4101, 1996.

[12] X. Godivier and F. Chapeau-Blondeau, “Noise-enhanced transmission of spike trains in the neuron,” Europhysics Letters,vol. 35, pp. 473–477, 1996.

[13] W. C. Stacey and D. M. Durand, “Stochastic resonance improves signal detection in hippocampal CA1 neurons,” Journal

of Neurophysiology, vol. 83, pp. 1394–1402, 2000.[14] B. Ando, S. Baglio, S. Graziani, and N. Pitrone, “Optimal improvement in bistable measurement device performance via

stochastic resonance,” International Journal of Electronics, vol. 86, pp. 86–87, 1999.[15] L. Gammaitoni and A. R. Bulsara, “Noise activated nonlinear dynamic sensors,” Physical Review Letters, vol. 88,

pp. 230601,1–4, 2002.[16] F. Chapeau-Blondeau and D. Rousseau, “Enhancement by noise in parallel arrays of sensors with power-law characteris-

tics,” Physical Review E, vol. 70, pp. 060101(R),1–4, 2004.[17] B. M. Jost and B. E. A. Saleh, “Signal-to-noise ratio improvement by stochastic resonance in a unidirectional photorefractive

ring resonator,” Optics Letters, vol. 21, pp. 287–289, 1996.[18] M. Misono, T. Kohmoto, Y. Fukuda, and M. Kunitomo, “Stochastic resonance in an optical bistable system driven by

colored noise,” Optics Communications, vol. 152, pp. 255–258, 1998.[19] S. Blanchard, D. Rousseau, D. Gindre, and F. Chapeau-Blondeau, “Constructive action of the speckle noise in a coherent

imaging system,” Optics Letters, vol. 32, pp. 1983–1985, 2007.[20] M. Ozer, M. Perc, and M. Uzuntarla, “Stochastic resonance on Newman-Watts networks of Hodgkin-Huxley neurons with

local periodic driving,” Physics Letters A, vol. 373, pp. 964–968, 2009.[21] M. Perc, “Stochastic resonance on excitable small-world networks via a pacemaker,” Physical Review E, vol. 76,

pp. 066203,1–7, 2007.[22] L. Martınez, T. Perez, C. R. Mirasso, and E. Manjarrez, “Stochastic resonance in the motor system: Effects of noise on

the monosynaptic reflex pathway of the cat spinal cord,” Journal of Neurophysiology, vol. 97, pp. 4007–4016, 2007.[23] E. Manjarrez, I. Mendez, L. Martinez, A. Flores, and C. R. Mirasso, “Effects of auditory noise on the psychophysical

detection of visual signals: Cross-modal stochastic resonance,” Neuroscience Letters, vol. 415, pp. 231–236, 2007.[24] M. Perc and M. Marhl, “Amplification of information transfer in excitable systems that reside in a steady state near a

bifurcation point to complex oscillatory behavior,” Physical Review E, vol. 71, pp. 026229,1–8, 2005.[25] D. C. Gong, G. Hu, X. D. Wen, C. Y. Yang, G. R. Qin, R. Li, and D. F. Ding, “Experimental study of signal-to-noise

ratio of stochastic resonance systems,” Physical Review A, vol. 46, pp. 3243–3249, 1992.[26] F. Chapeau-Blondeau and D. Rousseau, “Noise-aided SNR amplification by parallel arrays of sensors with saturation,”

Physics Letters A, vol. 351, pp. 231–237, 2006.[27] J. J. Collins, C. C. Chow, and T. T. Imhoff, “Aperiodic stochastic resonance in excitable systems,” Physical Review E,

vol. 52, pp. R3321–R3324, 1995.[28] F. Chapeau-Blondeau and J. Rojas-Varela, “Nonlinear signal propagation enhanced by noise via stochastic resonance,”

International Journal of Bifurcation and Chaos, vol. 10, pp. 1951–1959, 2000.[29] M. D. McDonnell, D. Abbott, and C. E. M. Pearce, “A characterization of suprathreshold stochastic resonance in an array

of comparators by correlation coefficient,” Fluctuation and Noise Letters, vol. 2, pp. L205–L220, 2002.

[30] M. E. Inchiosa and A. R. Bulsara, “Signal detection statistics of stochastic resonators,” Physical Review E, vol. 53,pp. R2021–R2024, 1996.

[31] V. Galdi, V. Pierro, and I. M. Pinto, “Evaluation of stochastic-resonance-based detectors of weak harmonic signals inadditive white Gaussian noise,” Physical Review E, vol. 57, pp. 6470–6479, 1998.

[32] M. D. McDonnell, D. Abbott, and C. E. M. Pearce, “An analysis of noise enhanced information transmission in an arrayof comparators,” Microelectronics Journal, vol. 33, pp. 1079–1089, 2002.

[33] F. Chapeau-Blondeau, “Noise-aided nonlinear Bayesian estimation,” Physical Review E, vol. 66, pp. 032101,1–3, 2002.[34] H. Chen, P. K. Varshney, and J. H. Michels, “Noise enhanced parameter estimation,” IEEE Transactions on Signal

Processing, vol. 56, pp. 5074–5081, 2008.[35] P. E. Greenwood, L. M. Ward, and W. Wefelmeyer, “Statistical analysis of stochastic resonance in a simple setting,”

Physical Review E, vol. 60, pp. 4687–4695, 1999.[36] Y. Wang and L. Wu, “Stochastic resonance and noise-enhanced Fisher information,” Fluctuation and Noise Letters, vol. 5,

pp. L435–L442, 2005.[37] F. Chapeau-Blondeau, S. Blanchard, and D. Rousseau, “Noise-enhanced Fisher information in parallel arrays of sensors

with saturation,” Physical Review E, vol. 74, pp. 031102,1–10, 2006.[38] A. R. Bulsara and A. Zador, “Threshold detection of wideband signals: A noise-controlled maximum in the mutual

information,” Physical Review E, vol. 54, pp. R2185–R2188, 1996.[39] G. Deco and B. Schurmann, “Stochastic resonance in the mutual information between input and output spike trains of

noisy central neurons,” Physica D, vol. 117, pp. 276–282, 1998.[40] N. G. Stocks, “Information transmission in parallel threshold arrays: Suprathreshold stochastic resonance,” Physical Review

E, vol. 63, pp. 041114,1–9, 2001.[41] D. Rousseau, J. Rojas Varela, and F. Chapeau-Blondeau, “Stochastic resonance for nonlinear sensors with saturation,”

Physical Review E, vol. 67, pp. 021102,1–6, 2003.[42] F. Chapeau-Blondeau, “Noise-enhanced capacity via stochastic resonance in an asymmetric binary channel,” Physical

Review E, vol. 55, pp. 2016–2019, 1997.[43] X. Godivier and F. Chapeau-Blondeau, “Stochastic resonance in the information capacity of a nonlinear dynamic system,”

International Journal of Bifurcation and Chaos, vol. 8, pp. 581–590, 1998.[44] L. B. Kish, G. P. Harmer, and D. Abbott, “Information transfer rate of neurons: Stochastic resonance of Shannon’s

information channel capacity,” Fluctuation and Noise Letters, vol. 1, pp. L13–L19, 2001.[45] P. beim Graben, “Estimating and improving the signal-to-noise ratio of time series by symbolic dynamics,” Physical Review

E, vol. 64, pp. 051104,1–15, 2001.[46] I. Csiszar, “Generalized cutoff rates and Renyi’s information measures,” IEEE Transactions on Information Theory, vol. 41,

pp. 26–34, 1995.[47] M. Ben-Bassat and J. Raviv, “Renyi’s entropy and the probability of error,” IEEE Transactions on Information Theory,

vol. 24, pp. 324–331, 1978.[48] L. L. Campbell, “A coding theorem and Renyi’s entropy,” Information and Control, vol. 8, pp. 423–429, 1965.[49] D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms. Cambridge: Cambridge University Press, 2003.[50] M. B. Baer, “Source coding for quasiarithmetic penalties,” IEEE Transactions on Information Theory, vol. 52, pp. 4380–

4393, 2006.[51] P. A. Humblet, “Generalization of Huffman coding to minimize the probability of buffer overflow,” IEEE Transactions on

Information Theory, vol. 27, pp. 230–232, 1981.[52] A. C. Blumer and R. J. McEliece, “The Renyi redundancy of generalized Huffman codes,” IEEE Transactions on Infor-

mation Theory, vol. 34, pp. 1242–1249, 1988.[53] M. B. Baer, “Optimal prefix codes for infinite alphabets with nonlinear costs,” IEEE Transactions on Information Theory,

vol. 54, pp. 1273–1286, 2008.[54] J. F. Bercher, “Source coding with escort distributions and Renyi entropy bounds,” Physics Letters A, vol. 373, pp. 3235–

3238, 2009.[55] E. Simonotto, M. Riani, C. Seife, M. Roberts, J. Twitty, and F. Moss, “Visual perception of stochastic resonance,” Physical

Review Letters, vol. 78, pp. 1186–1189, 1997.[56] D. Rousseau, A. Delahaies, and F. Chapeau-Blondeau, “Structural similarity measure to assess improvement by noise in

nonlinear image transmission,” IEEE Signal Processing Letters, vol. 17, pp. 36–39, 2010.