Sign up Calendar Latest Topics
 
 
 


Reply
  Author   Comment  
LawsonThompson

Registered:
Posts: 606
Reply with quote  #1 

We ran v2.7 at a convention this past weekend with a total of 27 missions and 211 crew members participating. We had 4 server crashes total, finding that if we restarted the server every 5 missions or so we could avoid a crash. The clients were rock solid, with only a bit of weirdness if a Fighter joined late.

In v2.6 I could sneak a peek at the Task Manager and see that if Artemis.exe had exceeded 1.5GB RAM, or if free RAM was below 1GB, it was time for a clean server restart. In v2.7, the server seems to hit 1.0GB RAM right away and hold steady. The only hint that a crash is imminent is if there is any lag on the main screen, and even then it might be so brief you don’t catch it. Artemis.exe does grow in size between every mission.

I have seen crashes both with and without the Artemis Bridge Tools hooked in, and have duplicated crashes on a pristine v2.0->2.7 upgrade. We are running on Windows 10 Home Edition, 64-bit on 8GB RAM, 1TB HDD, on a Core i3 7th generation (screaming fast single threading for the money) and an nVidia 1050Ti 6GB video card (overkill!).

I also just duplicated the crash on an i5 system with 16GB RAM, Windows 10 Home, SSD and nVidia 1070.

Our typical settings are:

  • Solo Game, Dreadnaught with an LR Shuttle and two Bombers.
  • Difficulty 5, Siege with Many Terrain, None Lethal Terrain, No Monsters, Many Friendlies, Many Anomalies. 
  • Packet interval is 20, 40, or 60; we tried all 3, thinking maybe the problem was network lag.
  • Time Limit: 30 to 60 minutes

We connect the server to a NetGear router over Ethernet, with Helm, Weapons, and Engineering also connected the same way, then all other stations to a Linksys 100Mbit 24 port switch. The NetGear is running OpenWRT to provide DHCP services, and pretty much nothing else significant.

The only crash pattern I’ve been able to determine:

  • The server always stays stable during the first few missions after a reset; I have never had it crash immediately after a fresh restart, even in 2.6.
  • This weekend, the crashes happened at least 10 minutes into a timed mission. It never crashes at launch or early in the mission.
  • It appears that when time expires, there is a chance of a crash. 
  • We run between 30 and 60 minutes; never more than 60 minutes.
  • I’ve crashed the server with no client connected, so I’m not entirely sure it’s network related.

By configuring a game as noted above, setting a 1 minute time limit, starting the game, connecting no clients, and pressing “E” to activate the automatic attract mode, the server will crash by itself after several missions. I can also trigger the crash by hitting ESC and clicking End Mission, then restarting the mission over and over again.

Is there any logging or diagnostics I can run? The most obvious pattern I see is having nVidia GPUs across all my server machines, though I'll try duplicating this on a standard laptop with Intel onboard and see what happens.


__________________
----
Visit us at http://www.ltebridge.com
Mike Substelny

Avatar / Picture

Administrator
Registered:
Posts: 2,346
Reply with quote  #2 
That's very interesting. I have been doing numerous experiments with if_scan_level (see other thread) and setting ship_text for generic objects. For those experiments I connect two networked clients running six stations and I run the GM console on the server machine. I have started a certain mission script dozens of times, let it run for 1-2 minutes, then stopped it. My server has stayed up for three days straight without even a reboot.

This makes me wonder if there is something in the Siege mission that causes the crash.

The mission script I've been using has no terrain. I wonder if the new multi-colored nebulae could be the problem(?).

Lawson, if you get the chance please repeat your experiment with no terrain and see if the crash still happens. I will add nebulae to my mission and see if I can get it to crash.

__________________
"The Admiralty had demanded six ships; the economists offered four; and we finally compromised on eight."
- Winston Churchill
MarkBell

Avatar / Picture

Administrator
Registered:
Posts: 1,935
Reply with quote  #3 
When you say Windows 10 Home Edition, is that with the Creator's Update from last year?  It would be version 1709 or newer.  I've heard it has some issues with DirectX, which sort of directly impacts Artemis as it stands.
__________________
Note - this is in no way intended to be an official position of Thom or Artemis, as I am not an official representative of the creator or game.
crashnburn68

Registered:
Posts: 18
Reply with quote  #4 
Quote:
Originally Posted by LawsonThompson

We had 4 server crashes total, finding that if we restarted the server every 5 missions or so we could avoid a crash. The clients were rock solid, with only a bit of weirdness if a Fighter joined late.



I'll try this restart method at my next All Night Artemis night. We had occasional crashes, and this logic lines up the best with what I experienced. I'll also try running it in different modes to see if I can replicate the crash.


LawsonThompson

Registered:
Posts: 606
Reply with quote  #5 
Quote:
Originally Posted by Mike Substelny
That's very interesting. I have been doing numerous experiments with if_scan_level (see other thread) and setting ship_text for generic objects. For those experiments I connect two networked clients running six stations and I run the GM console on the server machine. I have started a certain mission script dozens of times, let it run for 1-2 minutes, then stopped it. My server has stayed up for three days straight without even a reboot.

This makes me wonder if there is something in the Siege mission that causes the crash.

The mission script I've been using has no terrain. I wonder if the new multi-colored nebulae could be the problem(?).

Lawson, if you get the chance please repeat your experiment with no terrain and see if the crash still happens. I will add nebulae to my mission and see if I can get it to crash.


I've had this same problem on the Single Front mode, but we always set the Terrain settings to Many regardless.

I'll kick the settings around a bit and see what it does without terrain.

Maybe Artemis just gets bored with the generic settings I'm using?  [wink]

__________________
----
Visit us at http://www.ltebridge.com
LawsonThompson

Registered:
Posts: 606
Reply with quote  #6 
Quote:
Originally Posted by MarkBell
When you say Windows 10 Home Edition, is that with the Creator's Update from last year?  It would be version 1709 or newer.  I've heard it has some issues with DirectX, which sort of directly impacts Artemis as it stands.


Yes, the Windows 10 machines I'm using are fully updated with the Creator's Update. My server is a very new-ish system with a recent generation Core i3 and 8GB RAM. It maintains 40+FPS easily, though IMHO it should be more: I suspect the Creator's Update may have throttled something.

That said: on two identical Core i5-2520 laptops running side-by-side, Windows 7 vs Windows 10 frame rates are obviously different. Windows 7 runs at almost double the frame rate or more than Windows 10.

__________________
----
Visit us at http://www.ltebridge.com
crashnburn68

Registered:
Posts: 18
Reply with quote  #7 
Quote:
Originally Posted by LawsonThompson


Windows 7 runs at almost double the frame rate or more than Windows 10.


Lol, someone should *throttle* Microsoft.
Mike Substelny

Avatar / Picture

Administrator
Registered:
Posts: 2,346
Reply with quote  #8 
Quote:
Originally Posted by LawsonThompson
By configuring a game as noted above, setting a 1 minute time limit, starting the game, connecting no clients, and pressing “E” to activate the automatic attract mode, the server will crash by itself after several missions. I can also trigger the crash by hitting ESC and clicking End Mission, then restarting the mission over and over again.


I am attempting to replicate this experiment using exactly those conditions. My server has 4.00 GB of RAM. It is running Windows 10 Pro Version 1709 and my clients are running Windows 8.1.  I ran 30 consecutive one-minute missions without any sort of crash.

Artemis memory usage was slowly inching up, reaching 1,449.79 MB.

Then on my 31st mission my server locked up.

Fortunately Thom happens to be visiting my place right now. I described the problem. He will go home tonight and try to replicate the trouble on his dev machine and look for a solution.

__________________
"The Admiralty had demanded six ships; the economists offered four; and we finally compromised on eight."
- Winston Churchill
LawsonThompson

Registered:
Posts: 606
Reply with quote  #9 
Quote:
Originally Posted by Mike Substelny

...
Artemis memory usage was slowly inching up, reaching 1,449.79 MB.

Then on my 31st mission my server locked up.

Fortunately Thom happens to be visiting my place right now. I described the problem. He will go home tonight and try to replicate the trouble on his dev machine and look for a solution.


Feels odd to say this, but I hope it crashes for him! [wink] You can't fix what you can't replicate reliably. Random bugs are awful to troubleshoot.

At least this one does have a pattern--and you've helped convince me I'm not seeing things!

__________________
----
Visit us at http://www.ltebridge.com
ron77

Registered:
Posts: 158
Reply with quote  #10 
I ran a single Artemis 2.7.0 bridge at a local convention last weekend and found that the server did no crash once. Artemis on the comms station crashed once when the scenario spooled up and we had some very specific network problems with our science terminal that likely have nothing to do with Artemis, but other than that no major breakdowns. I did experience very noticable lag whenever we flew through an enemy fleet (several ships in close proximity being displayed on forward view at the same time), but that was likely due to our somewhat-sub-par graphics/video cards.
Previous Topic | Next Topic
Print
Reply

Quick Navigation:

Easily create a Forum Website with Website Toolbox.