Server.exe lock up (uses 100% of the process)
I have a project that runs every hour. It's been on the same server for a few months now and worked great. In just the last couple of days the server.exe will start to use 100% of the process. I'm using a .net application to connect to the project and run it. Even after closing the .net app, the process still runs at 100% (some times it will drop to 95 and then go back to 100).
If I restart the server and run the .net application again the server.exe is fine for several hours, then it will do the same thing. I've tried to reproduce the issue, but every time it runs fine for several hours and produces the correct output.
I've checked the log directory and nothing stands out. It's almost like the process is stuck in a loop. I'm confused as to why it would start now and repeatedly do this.
Is there something I can do to try and capture what is causing it to do this? For example advanced logging or something to see if it's in a loop and if so what project/script (or other) is causing this.
Solution found
Here are some details:
3 different systems, all windows:
Laptop:
Windows 7 Ultimate 64
Intel i7 1.60
4 GB RAM
NOTE: Screen-Scraper is installed as 32bit
Server 1:
Windows SBS 2008 64
Intel Xeon 2.53
4 GB RAM
Server 2:
Windows 2008 64 (Virtual)
Intel Xeon 1.87
3 GB RAM
Screen-Scraper Professional 5.5 (Released)
I found the reason for the lockup, which happens on all 3 systems. It appears to be an extractor pattern that should timeout, however never does. I have the timeout set at 30 seconds, however if I manually run a test on the pattern (a sub-extractor), it will take several minutes and the only thing that I see change is the amount of CPU usage increase. It only happens with one page so it was difficult to narrow down. (I should note that the process will move on, however I never receive the timeout dialog when test)
Each time the project runs, which is once per hour, this pattern will have an issue with one page. Instead of timing out, it will take an additional 10-15% of the CPU and after several minutes move on. After running for a few hours, the total CPU will peak at 95-100%.
This explains why it appeared to be random. It could also explain why the issue just started, if the page causing the problem is new, previous testing would not have found this issue.
I'll send an example file in so you can test it further.
What version of
What version of screen-scraper are you running? And on what OS?
By default screen-scraper is set for the most robust logging, and if you didn't set the logging to a lower level then you're seeing all that's put out. There are some methods you can use though. In you scripts, if you have any for or while loops you can add a session log that puts out sutil.getMemoryUsage, and that will help you go back and review where things started getting out of hand.