HARK 2.0 pre-alpha/code for BST notebook and dashboard #1055

MridulS · 2021-08-05T12:28:37Z

I went in a cleaned up a bit more from the Update-0p11-for-QE-revised branch and removed the boilerplate code files.
This one only has the changes made to the models.

continued from #1050

Didn't want to make changes to the active branches so created a new branch and the branch for this code is in Update-0p11-for-QE-revised-remove-sphinx.

sbenthall · 2021-08-05T13:23:44Z

Thanks @MridulS -- this will be a lot easier to review!

llorracc · 2021-08-05T13:37:18Z

@MridulS thanks. Is there a tool to do this specifically? If I had just deleted the sphinx files (and pytest-generated ones) it still would have wanted to replace the upstream files with my deletions. And I don't want to .gitignore those (esp the built sphinx docs) because when they are updated remotely I want to get them locally.

In your revised PR, it is much easier to see that almost everything falls into one of two categories:

ConsIndShockModel.py and core.py
Files like ConsINdShockModel_AgentDicts.py which are read in by the new (stub) ConsIndShockModel.py
Minor/cosmetic edits to jupyter notebooks

One small thing I was a bit confused about -- I merged in the changes Seb merged a couple of days ago, and it looked like Seb had removed my changes to DiscreteDistribution, where it inherited from DiscreteDistributionOld and then added new content. Since BST relies on the new version, I have reinstated that bit, but perhaps I misunderstood this part of Seb's merge.

sbenthall · 2021-08-05T13:50:32Z

One small thing I was a bit confused about -- I merged in the changes Seb merged a couple of days ago, and it looked like Seb had removed my changes to DiscreteDistribution, where it inherited from DiscreteDistributionOld and then added new content. Since BST relies on the new version, I have reinstated that bit, but perhaps I misunderstood this part of Seb's merge.

That sounds like an accurate description of changes I made to the 'innocuous changes' PR before merging.

I've opened #1051 for the pmf/pmv issue, which I think should be properly implemented in HARK, and #1053 to discuss the interface to calc_expectations.

llorracc · 2021-08-05T13:56:30Z

I see -- what I meant by "innocuous" was that they did not break any existing functionality. Adding new things -- especially essentially documentation -- also counts in my reckoning as innocuous. If there are other places where you removed functionality or documentation I added, we may need to restore those additions.

…

On Thu, Aug 5, 2021 at 9:50 AM Sebastian Benthall ***@***.***> wrote: One small thing I was a bit confused about -- I merged in the changes Seb merged a couple of days ago, and it looked like Seb had removed my changes to DiscreteDistribution, where it inherited from DiscreteDistributionOld and then added new content. Since BST relies on the new version, I have reinstated that bit, but perhaps I misunderstood this part of Seb's merge. That sounds like an accurate description of changes I made to the 'innocuous changes' PR before merging. I've opened #1051 <#1051> for the pmf/pmv issue, which I think should be properly implemented in HARK, and #1053 <#1053> to discuss the interface to calc_expectations. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1055 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKCK73WYDEECDTEDOUV6UTT3KJLFANCNFSM5BTYGKLQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

-- - Chris Carroll

sbenthall · 2021-08-05T14:08:10Z

These were the changes I made before I merged #1048
bc2051d

This was in accord with my review.
#1048 (comment)

I gathered merging this PR was urgent.

llorracc · 2021-08-05T15:12:01Z

Just tested it and this works with the public version of the BST notebook. Yay!

sbenthall · 2021-08-10T13:45:04Z

HARK/distribution.py

+# vector of point masses; here it is called the pmv (probability mass vector)
+
+
+class DiscreteDistribution(DiscreteDistributionOld):


To be addressed #1051

sbenthall · 2021-08-10T13:50:42Z

HARK/core.py

+        pseudo = (agent.pseudo_terminal is True)
+        if not pseudo:  # Then it's a real solution; should be part of the list
+            solution.insert(0, deepcopy(agent.solution_terminal))
+        completed_cycles = 0  # NOQA


This feature of being able to break off and restart a solution maybe can be split out and brought into HARK 1.0

sbenthall · 2021-08-10T13:51:06Z

HARK/core.py

+            if not go:  # Finished solving
+                # All models should incorporate 'stge_kind', but some have not
+                # Handle cases where that has not yet been implemented:
+                if not hasattr(solution_now.Bilt, 'stge_kind'):


what is "Bilt"?

It is a deliberate misspelling of "Built" - results that are constructed or might be modified in the course of the solution (as distinct from "Parm" which is parameters fed into the code).

PS. I know your preference is for words that are deliberately spelled correctly. A virtue of my choice of variable names is that if we can agree on a better alternative, a global search and replace will break neither normal text nor code.

sbenthall · 2021-08-11T14:36:45Z

HARK/core.py

+        to be tested.  Must have dict 'conditions' [name]
+
+    quiet : boolean
+        If True, construct the conditions but print nothing


I don't understand why there needs to be both a verbosity and a quietness setting.

sbenthall · 2021-08-11T14:39:23Z

HARK/core.py

+        # 1. use it pervasively
+        #    * in which case there should be a companion set_parameter
+        # or
+        # 2. Eliminate it


This is a good point.
I believe the issue is that the parameters dictionary was implemented for a few good reasons, but for backwards compatibility, it was made consistent with the 'whiteboard' way of accessing parameters through object attributes.

sbenthall · 2021-08-11T14:40:13Z

HARK/core.py

+        # 'approximation' pars
+        self.aprox_par = {'tolerance': self.tolerance, 'seed': self.seed}
+        # if lim exists, sol approaches truth as all aprox_par -> lim
+        self.aprox_lim = {'tolerance': 0.0, 'seed': None}


sbenthall · 2021-08-11T14:41:03Z

HARK/core.py

+            From logging.DEBUG to logging.CRITICAL
+
+        quietly : boolean
+            If True, suppress output


again.. if you want it to be quiet, couldn't you set the messaging level to something minimal?

There are some circs in which you may want things to run quietly, but not disturb the messaging level set by the user. It would be a pain (and error-prone) to set self.messaging_level_originally_requested_by_user then set the logging level to suppress all output, then to restore self.messaging_level_originally_requested_by_user. There are a couple of other use cases as well, like you might want to suppress output while debugging because you know what it will be and it just clutters things up.

a pain for who? error prone for who?
it seems natural for the method-level logging level to be scoped to the particular method, while leaving a global logging level in place as the default.

sbenthall · 2021-08-11T14:45:35Z

@llorracc you mention some handling of EOP shocks in this comment: #1039 (comment)

I've been looking for the corresponding changes in this PR but haven't been able to find them amid all the other changes.

I wonder if you could point to them directly.

sbenthall · 2021-08-12T15:46:53Z

HARK/ConsumptionSaving/ConsIndShockModel_AgentSolve.py

+            'chosen_to_next_BOP': {},  # or
+            'chosen_to_next_choice': {},
+            'EOP_to_next_BOP': {},  # or
+            'EOP_to_next_choice': {},  # EOP: End of Problem/Period


Support for BOP and EOP here:
#1039 (comment)

sbenthall · 2021-08-12T16:00:28Z

HARK/ConsumptionSaving/ConsIndShockModel_AgentSolve.py

+class Prospectations(SimpleNamespace):
+    """Expectations prior to the realization of current period shocks."""
+
+    pass


given the ambiguity about the timing of the shocks... maybe it would be better to have these namespaces be numbered, or indexed by the name of some variable, rather than named

sbenthall · 2021-08-12T16:02:38Z

HARK/ConsumptionSaving/ConsIndShockModel_AgentSolve.py

+             'BOP_to_choice': BOP_to_choice
+             }
+
+        return possible_transitions


a lot of structure is implied by the code here.
I wonder if it would be possible to require that the equations get laid out, and then have the structure (such as the timing of the shocks) be inferred from the model specification.
That's more along the lines of what I'm working on in #865

sbenthall · 2021-08-12T16:14:40Z

HARK/ConsumptionSaving/ConsIndShockModel_AgentSolve.py

+                    _log.info('\t' + str(Tran[key]['raw_text'][eqn_name]))
+
+    def check_conditions(self, messaging_level=logging.DEBUG, quietly=False):
+        """


I wonder if the check_conditions functionality is properly part of the solver functionality or part of the type defintion

sbenthall · 2021-08-12T16:18:53Z

HARK/ConsumptionSaving/ConsIndShockModel_Both.py

+
+    Bilt.__dict__.update({k: v for k, v in Modl.Rewards.vals.items()})
+
+    return soln


I am confused why a utility function (which knows its derivatives, which is an awesome addition!) returns a solution object.

Seems like a mathematical function is more basic than a solution.

This was part of a pattern of building up the contents of the solution object with standalone commands like

soln = self.def_utility_CRRA()
soln = self.def_value_funcs()

Probably would be an improvement to have self.def_utility_CRRA() return a 'utility' object, def_value_funcs a vFunc, etc. Then the calling code could become:

soln.u = self.def_utility_CRRA()
soln.vFunc = self.def_value_funcs()

The way these kinds of things are done right now is rather haphazard and inconsistent. An improvement in the code would be to systematize and regularlize these kinds of things.

sbenthall · 2021-08-12T16:31:00Z

HARK/ConsumptionSaving/ConsIndShockModel_AgentDicts.py

+init_perfect_foresight_plus.update(  # In principle, kinks exist all the way to infinity
+    {'aprox_par': {'MaxKinks': '500'}})
+init_perfect_foresight_plus.update(  # In principle, kinks exist all the way to infinity
+    {'aprox_lim': {'MaxKinks': float('inf')}})


sbenthall · 2021-08-12T16:32:12Z

HARK/ConsumptionSaving/ConsIndShockModel_AgentDicts.py

+CRRA_fcts.update({'prmtv_par': 'True'})
+init_perfect_foresight_plus['prmtv_par'].append('CRRA')
+# init_perfect_foresight_plus['_fcts'].update({'CRRA': CRRA_fcts})
+init_perfect_foresight_plus.update({'CRRA_fcts': CRRA_fcts})


Could you remind me what the different fcts are for?
Some seem to be about presentation.

Maybe a more explicit presentation layer is in order.

Perhaps a better alternative would be to use a python feature introduced in 3.9: Variables can now have documentation/explanations directly attached to them. Basically, seems to allow what I'm doing by adding _fcts, as built-in part of python.

I've long wondered why programming languages don't all allow this, rather than expecting users to hunt down potential explanations in the code. No good reason, I think.

sbenthall · 2021-08-12T16:33:39Z

HARK/ConsumptionSaving/ConsIndShockModel_AgentDicts.py

+init_perfect_foresight_plus.update({'T_age_fcts': T_age_fcts})
+
+T_cycle_fcts = {
+    'about': 'Number of periods in a "cycle" (like, a lifetime) for this agent type'}


maybe this can be inferred from the documentation

sbenthall · 2021-08-12T16:35:50Z

HARK/ConsumptionSaving/ConsIndShockModel_AgentTypes.py

+        As all aprox parameters approach their limits simultaneously,
+        the numerical solution should converge to the 'true' solution
+        that would be obtained with infinite computational power
+


sbenthall · 2021-08-12T17:23:22Z

HARK/ConsumptionSaving/ConsIndShockModel_AgentSolve.py

+            'permShk': shks_permuted[permPos] + zeros,  # + zeros fixes size
+            'tranShk': shks_permuted[tranPos] + zeros,  # all pmts for each st
+            'aNrm': starting_states
+        }


could the names of variables be extracted out for more generalizability?

sbenthall · 2024-06-03T18:01:12Z

This PR will never be merged into HARK, though it has been used as a source of inspiration to more recent work.
Can it be closed (to take it off the active PR's list)?

llorracc added 30 commits March 31, 2021 09:55

Should work after moving Ex to ConsumerSolution

7ff10c9

Before changing Stg to _0_

1ff940c

Before replacing mNrmMinNow with mNrmMin

be545e4

Seems to work

6f9466e

Seems to work; further cleanup is next

014d470

Before replacing soln_stge with curr

a088ed2

Works - Before nexting

e73e110

CRRA is both current and next; not allowed to be time varying

1656dce

Before removing nows

a7c7f02

After removing Now

9f8c719

Before replacing stg_cur.Nxt with stg_Nxt

416fc65

Comment out checks of passed equal next values

77032c8

Mostly comments in ConsIndShockModel.py

5853259

Incorporate edits to core.py

dc4d61d

cleanup

284488b

Eliminate duplication between IndShockSetup and PerfForesightSetup

f1db161

Before DiscFacEff -> DiscFacLiv

8e87f32

Before self.stg_crt.Nxt -> Nxt

ec090cb

Fixed OneState starting point

220e119

Runs without stopping

1ff5c86

Works after fixing recursion problems

5fc410c

Before __fcts to __fcts

3d74c5c

During Econ-ARK meeting w Mridul Seb

324c876

Before changing Nxt to stg_crt

d7db77c

Before moving fcts to Nxt

0ad31f8

stg_Nxt -> stg_futre

cc0979a

Finally seems to work -- but slows as it runs

b20f228

Fixed speed problems; cleanup

157b1a9

Before Ths -> tp1

f347581

Before moving check_* to PF

5a0dcb7

MridulS mentioned this pull request Aug 5, 2021

HARK 2.0 pre-alpha/code for BST notebook and dashboard #1050

Closed

sbenthall reviewed Aug 10, 2021

View reviewed changes

sbenthall reviewed Aug 11, 2021

View reviewed changes

sbenthall reviewed Aug 12, 2021

View reviewed changes

llorracc added 2 commits September 6, 2021 23:12

Fix bug in AgentSolve

59e9c63

Add more LaTeX definitions to utilities.py

3ba91db

sbenthall mentioned this pull request Nov 22, 2021

Syntax for mathematical expressions in model definitions #1085

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HARK 2.0 pre-alpha/code for BST notebook and dashboard #1055

HARK 2.0 pre-alpha/code for BST notebook and dashboard #1055

MridulS commented Aug 5, 2021 •

edited

Loading

sbenthall commented Aug 5, 2021

llorracc commented Aug 5, 2021

sbenthall commented Aug 5, 2021

llorracc commented Aug 5, 2021 via email

sbenthall commented Aug 5, 2021

llorracc commented Aug 5, 2021

sbenthall Aug 10, 2021

sbenthall Aug 10, 2021

sbenthall Aug 10, 2021

llorracc Aug 11, 2021

sbenthall Aug 11, 2021

sbenthall Aug 11, 2021

sbenthall Aug 11, 2021

sbenthall Aug 11, 2021

llorracc Aug 11, 2021

sbenthall Aug 11, 2021

sbenthall commented Aug 11, 2021

sbenthall Aug 12, 2021

sbenthall Aug 12, 2021

sbenthall Aug 12, 2021

sbenthall Aug 12, 2021

sbenthall Aug 12, 2021

llorracc Aug 12, 2021

sbenthall Aug 12, 2021

sbenthall Aug 12, 2021

llorracc Aug 12, 2021

sbenthall Aug 12, 2021

sbenthall Aug 12, 2021

sbenthall Aug 12, 2021

sbenthall commented Jun 3, 2024

		# vector of point masses; here it is called the pmv (probability mass vector)


		class DiscreteDistribution(DiscreteDistributionOld):


		Bilt.__dict__.update({k: v for k, v in Modl.Rewards.vals.items()})

		return soln

HARK 2.0 pre-alpha/code for BST notebook and dashboard #1055

Are you sure you want to change the base?

HARK 2.0 pre-alpha/code for BST notebook and dashboard #1055

Conversation

MridulS commented Aug 5, 2021 • edited Loading

sbenthall commented Aug 5, 2021

llorracc commented Aug 5, 2021

sbenthall commented Aug 5, 2021

llorracc commented Aug 5, 2021 via email

sbenthall commented Aug 5, 2021

llorracc commented Aug 5, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sbenthall commented Aug 11, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sbenthall commented Jun 3, 2024

MridulS commented Aug 5, 2021 •

edited

Loading