David's Astronomy Pages Notes - Session 998 (2022-04-02)

Notes (S997)

Notes Main

Home Page

Notes (S999A)

	Session Aims & Highlights
	- Observing Result  - Night Summary Plot  - Session Event Log
	Operational Issues
	- Critical Issues (0),  Major Issues (0),  Minor Issues (6),  Small Defects (0),  Continuous Improvement (1)
	Images from 2022-04-02 >>         [ Local Files >> ]

Session Aims & Highlights (2022-04-02)

Main aims

1. Targets.  Acquire images of a selection of variable stars, nearby stars, comets & deep sky targets as allowed by time & conditions.

Equipment & Software

• 12" LX200 and ST-10XME
• 80mm APO and ZWO ASI 178MC
• CCDApp2, TheSky6, CCDSoft5, DeviceHub 6.5.1.9,  MeadeGeneric 1.3.3.373, UPB 1.5, ASCOM 6.5 SP1
• Pegasus UPB & Motor Focus, TCF-S
• SharpCap 3.2, PHD2  2.6.10
   
• AstroMain 3.50.9,  AstroPlan 1.29.14,   AstroShCap 1.3.12, AstroGuard 1.7
  AstroVOE 1.9.2  AstroSN 1.7.3,  AstroProtect 1.6,  AstroMove 1.8.6
  (AstroWeather 1.16.5, AstroAllSky 3.15.3,  AstroObsCam 1.8, AstroLaunch 1.8.3)

Highlights

•  Targets.  15 targets succcessfully acquired
•  Dome/DeviceHub Glitch.  Some form of glitch occurred at 04:31:44 that led to Dome driver or DeviceHub to not report fresh information from the Dome. Although outgoing commands (Close Shutter, SlewToAzimuth(90) ) worked,  DeviceHub didn't show completion of dome closure or any rotation of the dome back to Park position (Az 90).

Notes:

• Session interrupted by rain/snow and cloud
  

  Summary Plots & Logs

Observing Plan

Observing Result

Dome & Scope Slewing Performance

Slew/Centering Performance

Guiding Performance Note:  Point (2) - Guiding Poor due to cloud (Dec Guiding - Black)

Sky Conditions (Locate Frames)

Night Sky Summary Plot Top axis: Sky Brightness at Zenith (in ADU/s) Lefthand axis: Local Time (hh LT). Righthand axis: Sun Altitude (degs)

Actual Weather vs Pre-Session Weather Forecast

Session Event Log

Time     Event Detail 22:17:00 Session Monitoring AutoStart monitoring for Live Session opportunity between 22:16 & 04:22 22:17:01 Session AutoStarting Session autostarting (22:17) 22:17:29 Session Created Live Session Created (2022-04-02 S00998, ImageSaveNum: 998001) 22:17:32   Scope Switched On Telescope Power has been switched on via UPB Powerbox. 22:19:22   Services Started Observatory Services started 22:19:28 Observatory (Auto) Observatory placed in Fully-Automated Mode 22:19:31 Session Pending Session pending (2022-04-02) 22:19:33 Session Initiating Session initiating (2022-04-02) 22:19:35   Plan Requested Observing Plan requested from AstroPlan (1.29.14) 22:20:06   Plan Loaded Observing Plan loaded to queue (Plan ID: 756) 22:20:26   Camera1 Connected SBIG Camera Connected (set point -25°C) 22:20:33   Telescope Connected Telescope Connected (TheSky6) 22:20:57 Session Equilibration Session ready for dome & camera equilibration 22:21:42   Dome Opened Dome opened (opening time 45s) 22:22:37 Session Running Session running 22:22:40   Queue Started Observing Queue started (22 targets selected) 22:22:43     Target Started (NrZen) Target started (Focus Field 10, HIP 49253) 22:24:28       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 22:27:40       Focusing Completed Foc1 AutoFocus Completed (Profile No 1, wide) 22:29:35       Focusing Completed Foc1 AutoFocus Completed (Profile No 1) 22:29:37       Focusing Started-Foc2 Foc2 Focusing Started (Secondary Scope, using ShCap) 22:31:49       Focusing Completed Foc2 AutoFocus Completed (Profile No 2, wide) 22:33:36       Focusing Completed Foc2 AutoFocus Completed (Profile No 2) 22:33:54     Target Completed Target completed (Focus Field 10, HIP 49253) 22:34:39     Target Started (1/22) Target started (1/22, AT2022evv (CV)) 22:37:53       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 22:39:55       Focusing Completed Foc1 AutoFocus Completed (Profile No 3) 22:45:42     Target Completed Target completed (1/22, AT2022evv (CV)) 22:46:39     Target Started (2/22) Target started (2/22, GCVS SY Cnc) 22:50:15       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S) 22:53:21   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars) 22:53:28     Target Completed Target completed (2/22, GCVS SY Cnc) 22:53:30   Queue Paused Queue paused for Session Suspension 22:53:31 Session Suspended Session suspended 22:54:21   Dome Closed Dome closed (closing time 50s) 23:02:22     Target Cancelled Target cancelled (3/23, NGC 4147, ToO) due to cloud 23:09:10     Target Cancelled Target cancelled (4/23, 19P/Borrelly) due to cloud 23:23:03     Target Cancelled Target cancelled (5/23, C/2019 L3 (ATLAS)) due to cloud 23:42:48     Target Cancelled Target cancelled (6/23, UGC 4219 w/AT2022eya) due to cloud 00:04:48     Target Cancelled Target cancelled (7/23, UGC 5560 w/SN2022ewv) due to cloud 00:16:36 Session Resuming Session resuming 00:17:21   Dome Opened Dome opened (opening time 45s) 00:17:23 Session Running Session running 00:17:25   Queue Resumed Observing Queue resumed 00:17:28     Target Started (8/23) Target started (8/23, NGC 4395 w/AT2022fmn) 00:33:39     Target Completed Target partially completed (8/23, NGC 4395 w/AT2022fmn) 00:33:43     Target Started (9/24) Target started (9/24, NGC 4147, ToO) 00:36:33       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 00:39:05       Focusing Completed Foc1 AutoFocus Completed (Profile No 4) 00:47:24     Target Completed Target completed (9/24, NGC 4147, ToO) 00:47:29     Target Started (10/24) Target started (10/24, UGC 5304) 00:54:10     Target Completed Target partially completed (10/24, UGC 5304) 00:54:14     Target Started (11/24) Target started (11/24, NGC 3367 w/SN2022ewj) 00:57:07       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S) 01:13:45     Target Completed Target completed (11/24, NGC 3367 w/SN2022ewj) 01:13:49     Target Started (12/24) Target started (12/24, NGC 5117 w/SN2022abq) 01:16:22       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S) 01:32:33     Target Completed Target completed (12/24, NGC 5117 w/SN2022abq) 01:35:53     Target Started (13/24) Target started (13/24, AT2022evu (CV)) 01:38:47       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S) 01:44:37     Target Completed Target completed (13/24, AT2022evu (CV)) 01:47:52     Target Started (14/24) Target started (14/24, GCVS R CrB) 01:50:35       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 01:52:28       Focusing Completed Foc1 AutoFocus Completed (Profile No 5) 01:55:49     Target Completed Target completed (14/24, GCVS R CrB) 01:57:09     Target Started (15/24) Target started (15/24, UGC 6312 w/SN2022eyj) 02:00:29       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 02:03:04       Focusing Completed Foc1 AutoFocus Completed (Profile No 6) 02:19:25     Target Completed Target completed (15/24, UGC 6312 w/SN2022eyj) 02:19:29     Target Started (16/24) Target started (16/24, IC 3862) 02:21:55       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 02:24:16       Focusing Completed Foc1 AutoFocus Completed (Profile No 7) 02:40:07     Target Completed Target completed (16/24, IC 3862) 02:41:09     Target Started (17/24) Target started (17/24, NGC 5427) 02:44:06       Focusing Skipped Foc1 focusing skipped - star is lost (TCF-S) 03:00:05     Target Completed Target completed (17/24, NGC 5427) 03:03:09     Target Started (18/24) Target started (18/24, NGC 5995 w/AT2022fqe) 03:06:16       Focusing Skipped Foc1 focusing skipped - star is lost (TCF-S) 03:22:14     Target Completed Target completed (18/24, NGC 5995 w/AT2022fqe) 03:25:09     Target Started (19/24) Target started (19/24, AT2021agpw (CV)) 03:27:24       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S) 03:27:30   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars) 03:28:30 Critical RainThreat Alert Critical Rain Threat Alert (Obs.Manager will close the Dome) 03:28:32   HardSuspend Called Hard Suspend is called due to Critical Conditions (threat of rain) 03:28:34     Target Aborted (19/24) Target aborted (19/24, AT2021agpw (CV)) due to threat of rain 03:28:36   Queue Paused Queue paused for Session Suspension 03:28:38 Session Suspended Session suspended 03:29:25   Dome Closed Dome closed (closing time 50s) 03:32:35 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 03:32 in section Monitor TheSky6 Scope 03:35:56 Session Resuming Session resuming 03:36:05     Target Cancelled Target cancelled (19/24, AT2021agpw (CV)) due to other condition - Blank 03:36:41   Dome Opened Dome opened (opening time 45s) 03:36:43 Session Running Session running 03:36:45   Queue Resumed Observing Queue resumed 03:37:09     Target Started (20/24) Target started (20/24, NGC 5732 w/SN2021agpf) 03:39:52       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 03:41:50       Focusing Completed Foc1 AutoFocus Completed (Profile No 8) 03:57:47     Target Completed Target completed (20/24, NGC 5732 w/SN2021agpf) 03:59:09     Target Started (21/24) Target started (21/24, GCVS S Cyg) 04:04:55       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 04:06:56       Focusing Failed Foc2 focusing check failed 04:06:58       Focusing Completed Foc1 AutoFocus Completed (Profile No 9) 04:07:30   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars) 04:12:31   SoftSuspend Cancelled Soft Suspend is cancelled due to improved conditions 04:15:16   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars) 04:18:21     Target Completed Target completed (21/24, GCVS S Cyg) 04:18:23   Queue Paused Queue paused for Session Suspension 04:18:25 Session Suspended Session suspended 04:19:12   Dome Closed Dome closed (closing time 50s) 04:27:38 Session Resuming Session resuming 04:28:23   Dome Opened Dome opened (opening time 45s) 04:28:25 Session Running Session running 04:28:27   Queue Resumed Observing Queue resumed 04:28:29     Target Started (22/24) Target started (22/24, GCVS SS Cyg) 04:31:26 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 04:31 in section Monitor Camera2 04:31:59 Program Resumed AstroMain UI resumed again at 04:31 after 10s 04:37:22     Target Completed Target completed (22/24, GCVS SS Cyg) 04:37:26     Target Started (23/24) Target started (23/24, GCVS BL Lac) 04:37:52     Target Failed Target failed due to slew error (23/24, GCVS BL Lac) 04:41:24     Target Started (24/24) Target started (24/24, M31 w/ATNova M31 2022-03c?) 04:41:42     Target Failed Target failed due to slew error (24/24, M31 w/ATNova M31 2022-03c?) 04:41:45   Queue Completed Job Queue completed 04:41:47 Session Closing Session closing 04:43:20   Dome Closing Dome closure hasn't finished after 90s (ShutterState=Closing) 08:19:56 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 08:19 in section Refresh Dome Picture 08:20:17 Program Resumed AstroMain UI resumed again at 08:20 after 12s 08:23:00 User Intervention User interventation to finishing session closing 08:23:07 Observatory (Manual) Observatory placed in Manual Mode 08:24:50   Telescope Parked Telescope parked (parking time 71s) 08:25:05   Telescope State Scope parked Turn scope off. (Handbox) 08:25:27   Telescope Switched Off Telescope Power has been switched off via UPB Switch. 08:28:57   Services Stopped Night Services stopped 08:32:33 User Intervention User request to close and finish session 08:32:35 Session Closing Session closing 08:32:37   Dome Closed Dome already closed (closed at 04:19, closing time 50s) 08:32:39   Dome Parked Dome already parked (parking time is unknown) 08:33:29   Services Stopped Night Services stopped 08:33:31 Session Housekeeping Housekeeping Started (Cleanup FITS, Create Fits Summary, Transfer Files) 08:33:44 Session Finished Session Finished

Session Alerts

Time     Alert Detail 22:35:25 Dome Discrepency Difference between Telescope Az and Dome Az is > 8 deg (AT2022evv) 03:33:05 Obs.Monitor Obs.Monitor appears to be frozen in section Monitor TheSky6 Scope (last cycle at 03:32:35) 03:33:35 Obs.Monitor Obs.Monitor has resumed again after 1.0 mins 04:31:58 Obs.Monitor Obs.Monitor appears to be frozen in section Monitor Focuser1 (last cycle at 04:31:26) 04:32:03 Dome Alert DomeRecords have stopped updating 04:32:06 Obs.Monitor Obs.Monitor has resumed again after 0.5 mins 04:32:08 Dome Alert DomeRecords have resumed updating 04:43:20 Dome Closing Dome closure hasn't finished after 90s (ShutterState=Closing) 04:43:50 Dome Discrepency Hub reports Dome is Closing, but Dome reports it is closed 08:20:27 Obs.Monitor Obs.Monitor appears to be frozen in section Refresh Dome Picture (last cycle at 08:19:56) 08:20:32 Obs.Monitor Obs.Monitor has resumed again after 0.5 mins

Back to Top

Operational Issues (2022-04-02, S998)

• Critical Issues (0)
• Major Issues (0)
• Minor Issues (6)
• Small Defects (0)
• Continuous Improvement (1)

[ Prev | Next ]

Critical Issues

• None

Major Issues

• None

Minor Issues

• TheSky6 showing information/error dialog reporting an 'mscorlib error' (04:36).  Checking-in remotely on the status of the session at around 04:30-04:35 showed a TheSky dialog window open that was titled 'mscorlib' and stated 'Object synchronization method was called from an unsynchronized block of code'.  (See Fig 1 below) No other information or details was available.  After screen capture the dialog was dismissed. This particular error dialog hasn't been seen before. It is unclear when the error dialog appeared - it could be anytime between 00:00 and 04:35.   If it occurred earlier it didn't seem to effect observatory/telescope operations at all. If it occurred later then it may have been responsible for apparent slew failures reported for at 04:37 (BL Lac) and  04:41 (M31 w/AT).  Log file shows that an exception with same description occurred at 04:31:44 in response to a get Dome.Connected property request at 04:31:36 suggesting that the TheSky dialog error probably occurred at this same time. It is proposed to put the issue down to a one-off unexplained glitch and close the issue for the time being.  Follow-up analysis shows Obs.Monitor froze for up to 1 minute at 04:31 with possible indication that Dome Records stopped updating briefly but both resumed at 04:32.   It is possible that issues related to the remote check-in made at 04:30-04:35 but it is unclear what user action would have caused the problems. 
  Closed 2022-04-03 (Session, TheSky, Microsoft Windows) 
    
• Obs.Monitor running very slowly at 04:31-04:32.  Obs.Monitor reported to have briefly frozen at 04:31:44. Following log log messages suggest Obs.Monitor is running slowly rather than is strictly frozen, since every the code section is which routine is frozen gradually changes with time. At about the same time, 'Object synchronization method was called from an unsynchronized block of code' exceptions occurred related to requests to Dome and to ?Scope.   It is unclear if the issue was triggered  by remote VNC  connection to check-in on the status of the session that was initiated at around 04:30 to 04:35 or is merely a coincidence.   UI and Obs.Monitor temporarily froze ay 08:129 to 08:20 around the time of Remote VNC connection to obsrevatory computer in the morning (connected ?).  Closed after review 2022-04-03.
  Closed 2022-04-03 (Session, AstroMain),
   
• Slew to final two targets (BL Lac and M31 w/AT) failed with AstroMain error 'Scope isn't pointing to the target'.  Checking-in remotely on the status of the session at around 04:30-04:35 showed a TheSky dialog window open that was titled 'mscorlib' and stated 'Object synchronization method was called from an unsynchronized block of code'.  Screen capture made at 04:36.  the dialog was dismissed (time not exactly known).   Analysis shows Obs.Monitor frooze for up to 1 minute at 04:31 with possible indication that Dome Records stopped updating briefly but both resumed at 04:32.   Issue couldn't have been caused by insertion of two ToO targets which happened at 04:40 which was well after 1st slew issue with  BL Lac.  Closed after review 2022-04-03 (unable to determine root cause).
  Closed 2022-04-03 (Session, DeviceHub) 
   
• Obs.Manager reporting discrepency between Shutter status from DeviceHub (closing) and the status reported in Pulsar Dome's ASCOM log (closed) (04:43). Discrepency recorded but didn't cause a stall to Obs.Manager due to past enhancements to code (cf code of Parking Dome).  When checked in morning and seeing  'Dome still closing'  dome shutter was checked using observatory's web cam and checked visually when showed that Dome was closed at at its normal park position). It was initially thought that this was a software failure of Pulsar Dome which was seen on and resolved by a dome hardware restart, however a successfully restar of Device Hub (which displayed Closed/Az 90° showed that the problem wasn't associated with Dome Control Unit but must be either due to an issue in DeviceHub software or in the Pulsar Dome' ASCOM Driver.   A discrepency noted between AstroMain/AstroGuard which showed shutter as closing and DeviceHub which showed shutter as Open, suggests an issue lies with DeviceHub. Analysis shows that Shutter successfully reopened at 04:28 following a short suspension (04:18-04:27). Issue closed after review 2022-04-03. 
  Closed 2022-04-03 (Session, DeviceHub, AstroMain) 
    
  Pulsar Log file shows that the regular sequence of Get requests from DeviceHub seem to stop at 04:31:23 or 04:31:24. with a single Volatile Exception being logged some 20s later  [ 04:31:44.178 : System.ApplicationException: Object synchronization method was called from an unsynchronized block of code.[0D][0A] at System.Threading.Mutex.ReleaseMutex()[0D][0A] at ASCOM.Utilities.TraceLogger.LogMessage(String Identifier, String Message) in C:\ASCOM Build\Export\ASCOM.Utilities\ASCOM.Utilities\TraceLogger.vb:line 424[0D][0A] at ASCOM.Pulsar_Observatories_Dome.Dome.Volatile() ]
  
  Pulsar Log file shows volatile messages (around 12 per second) thereafter with just the occasion Get Connected request.  CloseShutter request is seen at 04:41:50  and the volatile message show this completed at 04:42:35 (ie. a closing time of 45s).  
  
  Pulsar Log file shows a SlewToAzimuth 90 request at 04:43:55 and the dome rotated from Az 69.1 to Az 90.0 finishing the slew at 04:44:08  (ie 20.9 deg in 13s)
  
  Attempts to open dome as next session attempt (S999A, 2022-04-09) revealed that the Dome Rotation Unit had seemingly lost connection with the Shutter Unit. which might be related.  Graphs show that Battery Health data has been flat-lining since from 2022-04-05 14:00 (approx).
   
• Obs.Manager stalled during Park Dome operation whilst closing session (04:43). Command to Park Dome was effective (Dome physically moved at Az 90.0°) but DeviceHub was still reporting dome still slewing at Az 69.1°.  This followed directly after an issue during a closing shutter operation where DeviceHub.Dome was still reporting shutter status as 'Closing' (later 'Open') even though Shutter was physically closed ok.   Last log message was 'Starting DomeSlewDomeToAzimuth(90.0)' at 04:43:55.52. But we can tell that objDome.SlewToAzimuth(90) code line was executed since we can see the request reaching the Pulsar Log File.  Whilst the SlewDomeToAzimuth routine has a 3 minute timeout this would seem not to have been executed ?  (    may not be executed if slewing isn't determined to have started ?). Parking is performed by Dome.ParkDome2 routine which is run as a thread.  Obs.Manger proceeds in ClosingStageNo = 3 and waits for bParkDomeDone=True before proceeding to ClosingStageNo = 4 (ie. Scope Parking) .  It would seem that execution didn't return from the call SlewDomeToAzimuth(90, TaskError) since no ReportComment describing result was written to Report log.   One hypothesis is that after 3 minutes it entered the 'TimeOut Code' but execution held up at the AbortSlew call.   Issue resolved at 08:23 by a user intervention to place the Observatory back into Manual Mode.  A separate continuous improvement tickets has been raised to add a timeout checkout for the entire Park Dome operation. Closed after review 2022-04-03.
  Closed 2022-04-03  (Session, DeviceHub, AstroMain)  
   
• Scope left tracking between 04:23/04:31  (end of SS Cyg target) and 08:23 (user intervention to Park telescope).  Following Obs.Manager 'stall' during session closing at 04:23 the scope was left Tracking.  Although Obs.Manager Closing Session progress was held up it wasn't  frozen Obs Manager and it or ObsOverseer should have intervened to set ScopeTracking=False '. This didn't happen for some reason.   SetState_Closing() routine modified to make sureTelescope Tracking is Off.
  Fixed 2022-04-05 (AstroMain 3.50.10)    

Small Defects

• None

• Obs.Manager's Closing Session/Park Dome routine to be modifed to timeout after checking ASCOM Log data. Obs.Manager stalled during Park Dome operation whilst closing session (at 04:43). Operation whilst closing session (04:43). Command to Park Dome was effective (Dome physically moved at Az 90.0°) but DeviceHub was still reporting dome still slewing at Az 69.1°.  This followed directly after an issue during closing shutter where DeviceHub was reporting shutter as Closing (later Open) even though Shutter was physically closed ok.  Obs.Manager needs to have code added that allows the Parking Dome 'loop' to move-on ('exit')  when the DeviceHub data is showing a discrepency with Dome Az information that is being reported in the Pulsar Dome's ASCOM log. This would be similar to code used in Closing Shutter 'loop'.
  Can use DomeRecord.Slewing (integer) and DomeRecord.Azimuth (double) to check for discrepencies between Pulsar Log and DeviceHub.Dome.  Code added to Obs.Manager / ClosingState=4.
  Done 2022-04-04 (AstroMain 3.50.10)  

[ Prev | Next ]

Fig 1.  TheSky error dialog window  'Object synchronization method was called from an unsynchronized block of code'.

Back to Top