# Another wonky inverter: Solectria UMOC445TF breakdown



## TheSGC (Nov 15, 2007)

I have seen the encoder do all sorts of crazy things. You might have to check the DB9 connector for rust and open the back of the motor to check the encoder itself. 

I had to adjust the Encoder plate on my Force when the weather got cold out for the first time I had it, the car would jerk all over the place if it would move at all. Wolf has some good pictures on his site on the encoder plate and adjusting it wasn't too bad on a Acgtx20 motor. 

I do have a spare UMOC445 in my EV parts stash that is for sale if you are looking for a replacement/spare.


----------



## madderscience (Jun 28, 2008)

Hi TheSGC-

Thanks for the feedback. Do you have any idea if the AC55/UMOC encoder is a speed-and-direction encoder or a position encoder? If its the latter, then I can believe that problems with it or the connector could cause what I was experiencing. I've already got an extra UMOC so most likely I've got the spares I need if I choose to replace with original type parts.

Thanks


----------



## madderscience (Jun 28, 2008)

Hmm, the encoder is almost certainly a quadrature encoder with bipolar outputs, as it has GND, +6v, and A-, A+, and B-, B+ outputs. The last output on the pinout is TC+ which I beleive is the motor temperature.


I suppose if A or B were flaky the inverter might be confused as to the speed or direction of the motor, causing lurching I was experiencing. I will check connections and the condition of the connectors. These older solectria systems have DB9/DB25 computer style connectors on them which are NOT good in the weather. I've protected mine with silicone grease and an enclosure to help protect them, but still something to check.


BH


----------



## WolfTronix (Feb 8, 2016)

Step 1. Connect with hyper terminal (or other serial console program).
Step 2. See what the fault code is. 

The controller will typically tell you what is wrong.

If it is an encoder problem, you will see the displayed RPM jump around or just be wrong.

If it is a throttle signal problem, you will see the AD values not track the throttle position.

Etc..


----------



## WolfTronix (Feb 8, 2016)

Hyper Terminal settings:


----------



## madderscience (Jun 28, 2008)

Fault code is top of my list, hopefully the inverter remembers recent codes since the issue was intermittent. (It drove up behind the tow truck just fine of course)

"real men use minicom" 

Will report anything I learn to this thread. Thanks


----------



## madderscience (Jun 28, 2008)

I didn't have time tonight to get the wheels all the way off the ground but I got the car up on ramps and looked at the DB9 and DB25 plugs between the inverter and the encoder and control box. Both are still clean and appear in good shape. Nonetheless I plugged and unplugged each several times, then reassembled the protective housing around them.

I hooked up my laptop to the UMOC and watched its outputs while the car was sitting statically, so obviously yes I could not diagnose encoder goofiness that way but I was able to verify the various control signals (throttle, direction, limiter, etc) seem fine. Putting static torque on it (not enough to launch off the wheel ramps of course...) I saw basically equal current going to each of the phases.

I'll get the wheels all the way off the ground so I can spin them in the next couple days. I also decided that while awkward I can probably reach the encoder to remove it without taking the motor out thankfully, so I will take the cover off of it and inspect it. I might just replace the whole thing with the encoder off my other AC55 motor as a big-hammer solution. That one is from the Azure dynamics era and has an automotive grade connector on it. I'd have to update the connector on my umoc but that is totally doable.

BH


----------



## madderscience (Jun 28, 2008)

Update:

Today I got the wheels all the way off the ground and ran the car with a computer hooked up to monitor the inverter. All the various temperature readings were sane. The inverter temperature reading started at around 15c and went up a few degrees with me spinning the wheels. The encoder output was accurate and stable.

So, I took the car down off the blocks and drove it around. Initially just around the block repeatedly but it ended up being 35 miles, including hills, freeway, and retrieving my trailer from where it got left when the car went wonky. 

Zero problems of any sort, zero fault reports from the inverter.

The hottest I got the inverter was about 31c, or about 15c warmer than ambient temperature. I was intentionally driving aggressively to try and make something happen, if it was going to, so inverter temperature was almost certainly at least several degrees warmer than when the breakdown occurred. Weather was similar to the day the problems happened, maybe a few degrees cooler and less humid, but still drizzly/wet.

All I have done to it is reseat the encoder (DB9) and control signal (DB25) plugs. Maybe that is all it needed.

At this point I can't imagine it is any problem with an IGBT module. If there is a problem with the driver board heating it up didn't cause the problems. 

I took the encoder off of my SPARE ac55 just to see how it is made. It is a potted/sealed module and it uses hall effect sensors, not optical, so should be pretty robust presuming identical to the one in the car. There are no contacts or plugs (except for the motor temperature sensor) within the motor to come loose or otherwise lose contact.

I'm not quite ready to start driving it to work again but I will start driving it for errands again and build up some more confidence.


----------



## madderscience (Jun 28, 2008)

Hmm, another 2 days and 70-ish miles of driving, half of it in pouring rain, and zero faults.

At this point I am cautiously optimistic that the issue was indeed contact problems between the encoder and the inverter at the DB9 plug. This is despite the plug protected by an outer water resistant enclosure, and coated in silicone grease.

I'm thinking my reaction at this point should be proactive:

1) reposition the enclosure with the encoder and control box plugs to someplace I can reach from the top of the car
2) finish the project started long ago of getting the onboard computer hooked up to the inverter's serial output and data logging to catch future faults.
3) proactively rebuild my spare UMOC.

BTW while I don't think it itself is the problem, for future reference the encoder on an AC55 for the UMOC has part number 7200-0944. I was unable to find such an item online, but there is this:

http://www.contrexinc.com/index.php/products-overview/sensor-kits/speed-sensors

The Contrex 7200-0943 sensor appears to be the same thing (differential output quadrature encoder, same voltage range, same number of teeth) I think the only difference might be the 0944 has 2 extra wires for the motor temperature sensor. That's just a guess however. if anybody knows for sure the info would be appreciated.


----------



## madderscience (Jun 28, 2008)

I contacted contrex, inc asking them if they did make a custom encoder for Azure.  Suprisingly they replied and stated that they did.

They said it was identical to their 7200-0992 model, but with 14" wires. That doesn't match the wire length on what I have, but its a differential quadrature encoder so it could be the same beast.

There are AC55s out there with a different encoder that were meant to be used with a different manufacturer's inverter. If anybody has one of those, this might be the info needed to convert it over to run with a UMOC/DMOC.

My own AC55/UMOC still going. slowly rebuilding confindence, but still driving with the laptop hooked up hoping to catch it in the act.


----------



## madderscience (Jun 28, 2008)

Gaa-

This is not the thread I would resurrect if I had the choice.

My UMOC 445TF has been working fine since the last post on this thread - up until late november. (now at 69000+ EV miles)

I was driving home on freeway, and sudden inverter shutoff. No signal lights or serial output. I was able to coast to next off ramp (yay LRR tires) but once safely stopped I could not get the inverter to reset. On power up it appeared to be in a tight reset loop. (various 5v signals like brake relay, etc cycled on and off rapidly)

For about 10 minutes prior to the failure I was seeing telemetry from the inverter (which I collect near real time - a feature added after the last failure) indicating "OverCur B" - which the UMOC manual says nothing about but a DMOC645 manual I have suggests this is overcurrent on phase B of the motor. I Imagine its the same thing for the UMOC. Despite the warning the car was driving fine til the sudden failure. 

Like last time, after the tow of shame the car powered up and I was able to drive it a few feet into the garage.

FWIW here is the full telemetry (my data format, but variable names come from the UMOC serial output) for the last sample collected before the failure:

"Inv": {"battv":254, "d":36, "errPal":"OverCur B", "lastErr":"", "limBatV":0.8571, "limBoxTmp":1.0, "limDevTmp":0.8571, "limMtrTmp":1.0, "limRpm":0.8571, "pwrSav":1.0, "pwrStage":"ENABLED", "q":-103, "regen":"ENABLED", "relay":"ON", "rpm":3464, "temp":24},

I'm going to go through the same drill that was recommended and I did last time - reseat all plugs, check for any imbalance in phase currents, etc. I'm also going to open the inverter up (first time in 6 years) and check things out in there, re-seat plugs, etc. 

Since last time (and perhaps this time too) it turned out to be the DB9/DB25 connectors I have finally repositioned the junction box that protects them so I can get at them from the top of the car, allowing a field repair in the future.


----------



## madderscience (Jun 28, 2008)

Bumping my ancient thread again 

So an update to respond to my last post on my UMOC445. I know this is long but a lot of useful data points here for anybody else running solectria stuff, especially as the miles start piling up...

The car was taken down for substantial overhaul after the late november inverter failure, including lots of things unrelated to the inverter but which ere piling up - repair of body damage after being sideswiped by a moving truck - which required pulling the battery - overhauling the battery, consisting of replacing a bunch of broken tension bands (I messed up manufacturing them is main reason for failures), replacing 2 weakest cells with my remaining spares, and cleaning and re-torquing all the bus bars the elithion BMS had been reporting high cell resistance or high heat on. I also had to fabricate the right side half shaft as the custom one that was made back in 2013 by a race shop in Georgia fell apart during disassembly of the car, and I was unable to get it to stay together again. Its rubber boots were also near failure. Also did other stuff like suspension, brakes, etc.

But about the inverter:

My inverter in the car is UMOC 445 S/N #448. I took it apart for the first time since getting the car running in 2013. Back in 2013 all I had done was clean some very slight corrosion (white dusty stuff) off the circuit boards, check torque on all the major power section connections, and reassemble. 

This time around, I planned to start with the same thing. Here is what I found (and remember, this inverter now has approximately 69K miles on it, plus whatever it may have seen before I got ahold of it, but probably not much more:

1) Ribbon cable from main computer board to bottom board (gate driver board) had been rubbing on top of CPU to the point where several conductors were exposed. I don't think anything shorted, but obviously not good. This due to mechanical vibration no doubt. Fixed by wrapping a band of good quality electrical tape around the ribbon cable at that point.

2) Nearly broken solder connection on main computer board - one of the resistors - no idea its purpose - seated in a plug instead of soldered in - the plug terminal was nearly broken off from the motherboard, probably originally damaged by rough/clumsy handling at some point. I soldered it back down, and soldered in all the other plugged resistors. I can't imagine a reason to ever swap them out.

3) A mounting screw had rattled loose on the outside of the UMOC case, and its screw-thread insert had fallen into the inside of the inverter and was rattling around therein. Fortunately since the power section is upside down, the insert was rattling around on the bottom of the case, but it could easily have bounced onto the CPU board and caused all kinds of havoc. I pressed it back into place and made sure the screw on the outside (holding one of the two mounting angle brackets) was tight.

4) Mounting screw for the serial terminal DB9 was loose. Tightened.

---

With these fixes, completed in early March 2020, the car was driveable again, but I found I was having problems with regen causing the BMS to fault and in turn, kick the car into 'safe' mode (current limited, no regen). The inverter itself would also periodically, but much more rarely, fault. The inverter fault when it occurred indicated excessive battery voltage.

At first I decided that the problems with regen may be due to differences in cable routing or some other effect of having the whole battery apart and back together again, so I made a much of improvements to grounding, shielding, noise filtering, etc. on the BMS power supply and did things like add some filter capacitors at the battery outputs to try and clean up inverter noise before it got to the batteries. This did manage to reduce the probablility of regen triggering a fault but it didn't go away. I also turned the regen power down to about half what it had been before all this started. Having not succeeded in cleaning up noise I began to suspect the problem was with the inverter itself, especially since I was still seeing periodic inverter faults at a much higher rate than I had ever seen before.

I continued to drive the car like this up until late April.

---

In late april, over the course of a few days, the inverter went from flaky but usable to unusable. The failure mode was the dreaded 'DESAT" error, which is the inverter claiming to detect a fault with the IGBTs - either a short circuit or no current flowing when it expects it. Not sure of the details, but basically its a severe error. However, if I pulled over and reset the inverter, I could drive for a few minutes before it happened again. So whatever the problem was, it was not an exploded IGBT or other major failure. I managed to limp the car home and ignored it again for a couple weeks.

I decided to replace all the major power section components: main filter caps, smaller filter caps, and the IGBTs. I was suspecting that the problem might be the main filter caps, now with 50000+ hours on them and likely around 50% of the way through their useful rated life given the usage profile they see, may be starting to fail. (these are 20 year old electrolytic capacitors - good quality manufactured, but getting up there in age). For the record, newer, better parts are available now which I was able to find:

Main electrolytic capacitors upgraded to superior newer design. Higher voltage rating, about 25% greater capacitance, lower series resistance, higher ripple current rating:

Originals: 400V 2200uf rated. Nippon chemi-con U36D series. ESR (equivalent series resistance) 75mOhm, ripple current 6.3A
Replacements: 450V 2700uf rated Nippon chemi-con U37F series part number E37F451HPN272MCB7M. ESR 34mOhm 9.8A ripple current. Mouser Stocked. Identical can size and terminal geometry to originals.

Smaller film type filter capacitors: 
Originals are 400v, 2uF rated axial lead 2 paralleled per IGBT 
Replacements": 600V 4uF rated axial lead type Illinois Capacitor / 405PHC600K (digi key stocked) These are much larger physically than the originals but only one is needed per IGBT and still fits underneath main bus bars between the IGBTs.

IGBTs: UMOC #448 has mitsubishi CM400DU-12F modules. these are 600V, 400A rated half bridges. 3 per UMOC. These are obsolete parts, but appear to still be available new surplus from various sources. My new ones came via ebay from a Tesla research facility apparently....

I also replaced the main filter capacitor on the driver board power supply - old rating 450V 22uF, new 450V 33uF.

When taking the inverter apart for a second time, I did a (low voltage) test on all the power section capacitors - all in spec. I didn't try a high voltage test on them since I was going for the shotgun method of replacing everything in sight.

When disassembling the power section, I think I found what was likely the REAL PROBLEM ALL ALONG. As I was loosening the bolts holding the positive bus bar, which also hold down ring terminals to the smaller filter capacitors and positive voltage reference to the IGBT drivers, the phase "B" IGBT driver voltage reference wire broke off at the ring terminal. The wire had corroded inside the insulation. It was likely making only intermittent contact and this failure would make the driver unable to properly control that IGBT, and that probably triggered the intermittent but increasing-in-frequency DESAT error. So I checked all the other pigtails coming off the driver board and going to the contactor. Every other wire looked fine. Nonetheless I replaced all three IGBT driver high voltage pigtails. I should note that the failure error I captured way back in november when this all started was an error on phase "B" - same IGBT that had the corroded wire to its driver. 1 in 3 chance but still, good that it correlates.

Having done all that, and as carefully and neatly as I could reassembling the inverter. I got it back into the car today and drove it about 15 miles with zero faults, regen or otherwise. That is way better than it was at the end, though I cannot yet definitively say everything is fixed. That will require a couple more weeks of confidence building, some freeway driving, and turning the regen back up to where I originally had it. I also now have all the capacitors in the power section with at least a 450V rating, whereas before many were 400V rated. So a little more safety margin there, though I do not intend to try to increase my pack voltage. I know there are lots of other considerations besides just the capacitors to being able to do something like that.


Note: My only slightly older by S/N spare UMOC #442 has a DIFFERENT OLDER INFERIOR IGBT MODULE in it - it has Mitsubishi CM400DY-12H part number half bridges in it. Looking at the data sheet, these have forward voltage drop a few tenths of a volt higher than the CM400DU-12F modules. Doesn't seem like much but that can be hundreds of watts more heat that the inverter has to dissipate. I have never had problems with overheating on my UMOC, but the older modules would cause more heating. So it appears that someplace between S/N 442 and 448 Solectria made a design improvement to their power section. It appears that there is no difference to the driver board, and mechanically it looks like it is possible to upgrade an older UMOC with the newer modules, even though mechanically they are different (newer modules have their main lugs offset, older are centered, this moves the whole bank of capacitors over about 15mm.


----------



## kennybobby (Aug 10, 2012)

madderscience said:


> ...
> 
> 1) Ribbon cable from main computer board to bottom board (gate driver board) had been rubbing on top of CPU to the point where several conductors were exposed.
> 
> ...


Great troubleshooting and repair report, plus good catch on the corroded ring terminal. Thanks for sharing.


----------



## madderscience (Jun 28, 2008)

thanks for summarizing my overly long post 

The other thing with these UMOCs to check first on any kind of weirdness is the condition of the DB9 and DB25 plugs for the motor encoder and inverter control signals respectively. These plugs are computer grade plugs never intended for out-in-the-weather operation. I use silicone dielectric grease on the terminals of mine to help protect them from corrosion and have an enclosure box that contains all the plugs and junctions to help protect them from weather, but its still a good policy to check these plugs and re-seat them at least once or twice a year I'd say.


----------

