[meteorite-list] Mars Rover Opportunity Suffers Worrying Bouts of 'Amnesia'

[Ron Baalke]

http://news.discovery.com/space/mars-rover-opportunity-suffers-worrying-bouts-of-amnesia-141229.htm

Mars Rover Opportunity Suffers Worrying Bouts of 'Amnesia'
by Ian O'Neill
Discovery News
December 29, 2014

Problems with NASA's Mars Exploration Rover Opportunity's flash memory 
have intensified over recent weeks, so Discovery News Space Producer Ian 
O'Neill spoke with NASA Project Manager John Callas about the severity 
of the glitches, how they're affecting Opportunity's mission and how his 
team hope to find a fix.

Mars Exploration Rover Opportunity has been exploring the Martian surface 
for over a decade - that's an amazing ten years longer than the 3-month 
primary mission it began in January 2004. But with its great successes, 
inevitable age-related issues have surfaced and mission engineers are 
being challenged by an increasingly troubling bout of rover "amnesia."

Opportunity utilizes two types of memory to record mission telemetry as 
it explores the Meridiani Planum region. Sister rover Spirit, which sadly 
succumbed to the Martian elements in 2010 after 6 years of exploring Mars, 
used the same system. The two types of memory are known as "volatile" 
and "non-volatile."

"The difference is non-volatile memory remembers everything even if 
you power off, in volatile memory everything goes away," said Mars Exploration 
Rover Project Manager John Callas, of NASA's Jet Propulsion Laboratory 
in Pasadena, Calif. "So volatile memory is like the traditional RAM 
you have in your computer; non-volatile memory uses flash memory technology."

Usually, all telemetry data is stored in the flash memory, so that when 
the rover powers down during the Martian night or reboots, the data remains 
stored - like when you turn off your digital camera, the photos remain 
saved to the camera's flash card. Any data stored in the rover's RAM, 
however, is lost as it shuts down.

Flash memory may be great for storing data when the rover's electronics 
are powered down, "but flash memory has a limitation on how many times 
you can read and write to it," Callas told Discovery News. "It 'wears 
out' with use."

And, after a decade of continuous use, it's the rover's flash memory 
that mission engineers have identified as the source of lost data and 
unexpected reset events that are plaguing the rover's surface mission.

Old Rover Memory Glitches

"The problems started off fairly benign, but now they've become more 
serious - much like an illness, the symptoms were mild, but now with 
the progression of time things have become more serious," added Callas.

"So now we're having these events we call 'amnesia,' which is 
the rover trying to use the flash memory, but it wasn't able to, so 
instead it uses the RAM - it stores telemetry data in that volatile 
memory, but when the rover goes to sleep and wakes up again, all (the 
data) is gone. So that's why we call it amnesia - it forgets what 
it has done."

Opportunity uses NASA's veteran Mars Odyssey satellite as a communications 
relay between the Red Planet and Earth, so whenever Odyssey makes an orbital 
pass, commands are sent down to the rover and telemetry is beamed back 
to Earth.

But should an orbital pass be unavailable until the rover has powered 
down and then rebooted the following day, the rover team noticed that 
data was being lost - the rover had been encountering the flash memory 
error and then saving it to RAM, avoiding the flash memory all together. 
As the rover powered down, the RAM was wiped and the data was gone the 
following day.

Christmas Blues

The flash memory issue has grown into a bigger problem than losing valuable 
data, however.

As the rover attempts to save data to the flash memory, and is repeatedly 
unsuccessful, its software forces the rover to reboot. If a sequence of 
commands is sent to the rover, it will keep rebooting over and over again, 
forgetting what the previous command instructed the rover to do.

"Basically the rover stops what it was doing because it wasn't sure 
what caused the reset," said Callas. "So that interrupts our science 
mission on the surface of Mars.

"It's like you're trying to drive on a family trip - the car stalls 
out every 5 minutes. You don't make much progress that way!"

And now for the rover team's worse nightmare has reared its ugly head 
- Opportunity stopped communicating with Earth over the Christmas break.

As the NASA team went into the Christmas holidays, a series of 3 sol (Mars 
day) plans gave the rover a sequence of commands to work on. On the first 
sol, the rover would operate as expected, but come the second and third 
sols, not only would the rover not execute the rest of the commands, it 
stopped talking to mission control.

Like any space mission, when Opportunity stops talking, "we get very, 
very worried," said Callas.

Fortunately, after sending commands to the rover, it sent a reassuring 
"beep" in reply and continued with its instructions.

Software Fix?

It seems the source for all these problems lead back to one particular 
bank of flash memory. 7 banks are used by Opportunity and it's the 7th 
bank that is triggering the data loss, rover resets and communications 
glitches.

Now the culprit has been identified, JPL software engineers have developed 
a technique that will force the rover's software to ignore the 7th bank 
and utilize the other 6 apparently healthy banks. According to Callas, 
his team is probably a couple of weeks away from completing the software 
change so it can be uploaded to Opportunity.

Apart from this troubling turn of events, Callas is amazed at the health 
of the rest of the rover's hardware, but he remains realistic about 
Opportunity's future.

"The rover has been amazingly healthy considering how much we've used 
it - we thought the mobility system would have worn out a long ago but 
it's in great health.

"But anything could fail at any moment," he said. "It's like you 
have an aging parent, that is otherwise in good health - maybe they 
go for a little jog every day, play tennis each day - but you never 
know, they could have a massive stroke right in the middle of the night. 
So we're always cautious that something could happen."

And the kicker, says Callas, is that Opportunity's most exciting science 
could be less than half a mile from the rover's current location at 
the rim of Endeavour Crater.

A Sprint to Marathon Valley

"Perhaps the most exciting part of the mission is ahead of us...we 
have this valley, we call it Marathon Valley, only about 650 meters away 
from the rover."

Marathon Valley is so-called as the location marks the distance the rover 
will have exceeded a marathon on Mars should it get there. Opportunity 
has traversed over 26 miles and currently holds the off-world record for 
any rover - robotic or driven by an Apollo astronaut.

According to orbital mapping of Marathon Valley, the location contains 
a variety of clay minerals that could have only been formed when Mars 
had an abundance of pH-neutral water on its surface. It has ancient geology 
spanning back to the Noachian era, much older than Gale Crater - where 
NASA's Curiosity rover is currently exploring. Like Opportunity's 
previous exploration of clay-rich deposits, studies of Marathon Valley 
could provide invaluable data as to the ancient, potentially habitable 
Mars environment.

So as we keep our fingers crossed for a successful software fix for Opportunity 
and the huge science potential the rover still promises, it's mind-blowing 
to think that this rover, that had a primary mission of just 3 short months 
in 2004, is still doing incredible science in this alien environment, 
adding more pieces to the Mars habitability puzzle.