Difference between Two Challenges in Malaria RL

#1

Hi Everyone,

I was hoping if someone could direct me. I wanted to know what is the difference between ChallengeSeqDecEnvironment and the ChallengeEnvironment Class in the netsapi library.

Thanks a lot for your help.
Apoorv

0 Likes

#2

Hello Apoorv!
In the ChallengeSeqDecEnvironment you have the ability to evaluate an action (a pair of interventions) at a given state in sequence, and also to evaluate an intervention policy (5 actions evaluated in sequence).

In the ChallengeEnvironment you can only evaluate policies, and these policies are more limited than in the previous environment. In essence, the policy is to perform one (1) action in the first year, and then perform no further actions (i.e. all remaining interventions have zero coverage). This environment was used as a simple setting to explain the goals of the challenge in the tutorials.

0 Likes

#3

Would also like to add that using ChallengeSeqDecEnvironment is advised based on updates and the example solutions in the tutorials.
(Plus, your scores can be higher!)

1 Like

#4

Hello sekou!

Essentially what I understand is we can get reward values after each intervention (action) in the case of ChallengeSeqDecEnvironment but in the case of ChallengeEnvironment we would not be getting any of the intermediate rewards, only a cumulation of them at the end of the episode (end of 5 years of intervention).

Am I correct to understand it like above?

Thanks a lot for your help!
Apoorv

0 Likes

#5

Slight correction, you can get reward values after each intervention (action) or at the end of the episode in the case of ChallengeSeqDecEnvironment. There are two evaluate methods for this environment.

I’d say don’t use the ChallengeEnvironment, because its episodic reward is only based on the action performed in the first year. This environment is still around only because it was used before the challenge launch to explain the concept.

1 Like

#6

Got it! Thanks a ton @sekou. :smiley:

0 Likes