This text was ready by Kevin Jackson for the annual Supercomputer Convention; SC24 can be held in Atlanta from Sunday, Nov. 17 to Friday, Nov. 22.
Science lies on the coronary heart of the annual Supercomputing convention, and the Technical Program is without doubt one of the most vital and difficult points of the convention. To be taught extra about what this program does, in addition to the scientific imaginative and prescient that drives each resolution inside the program, right here’s an interview with SC24 Technical Program Chair Guillaume Pallez (Inria) and Vice Chair Judith Hill (LLNL).
Q: Are you able to inform us about your position as SC24 Technical Program Chair?
TP: Everybody might be conscious of the breadth of the SC Technical Program.
The seen half to most attendees contains the Technical Papers, Workshops, Tutorials, Panels, Posters, and Birds of a Feather. This 12 months we even launched a brand new observe referred to as Art of HPC. The Technical Program additionally contains SC-specific awards akin to Take a look at of Time, and we coordinate with the computing societies (ACM, IEEE, SIGHPC, TCHPC) for his or her particular awards and the award ceremony.
Then there are the much less seen components, that are simply as vital. These embody the reproducibility analysis for the Technical Papers, in addition to the coordination with the SC Pupil Cluster Competitors to pick the paper from final 12 months that can be reproduced by the SCC groups. We additionally handle the proceedings for papers and workshops in cooperation with IEEE.
Our first position is guaranteeing that the entire course of round and between these components is nicely coordinated. This implies not solely offering a wonderful scientific occasion to attendees, but additionally guaranteeing that in SC, folks can navigate from one factor to a different in a stress-free surroundings.
Our second position is to supply a world scientific imaginative and prescient and oversight over the Technical Program of SC.
For all of those components, we’re supported by the respective factor chairs and vice-chairs. With out their exhausting work and dedication to SC, it will not be potential to host an occasion of the dimensions and scope of the SC Technical Program.
Q: What do you imply by offering a “scientific imaginative and prescient?”
TP: Main the Technical Program of SC is a chance to have an effect on our area, and on the science that we need to see. We consider that on this position now we have a accountability to the group to enhance our scientific course of.
For SC24, there are three components that we need to spotlight:
- Offering a key venue the place attendees have entry to the variety of the science that’s achieved round HPC. This contains matters the place HPC is the secondary scientific factor or an enabling know-how, and has traditionally not been submitted to SC.
- Ensuring that by doing this, we don’t degrade the standard of the technical program (i.e. we wish the highest paper from these matters).
- Lastly, an vital factor is the perfect practices that our group ought to subscribe to, and SC’s management position in establishing these.
Q: Are you able to give us examples of what you imply?
TP: To additional diversify the SC Technical Program, we augmented two components to draw further submissions that won’t have in any other case felt welcome at SC:
- Workshops present a singular alternative to deal with particular scientific matters with early concepts that will not be mature but, however that may be brainstormed with a set of colleagues. As well as, there is a chance to debate matters which might be fascinating to the SC group, however the place the principle outcomes have a tendency to not be submitted at SC (i.e. outcomes that may be printed in excessive rating conferences from different scientific domains). For these matters the place SC will not be their main venue for publication, now we have tried to encourage proceedings-free symposiums, i.e. specialised occasions the place submissions are the most recent key outcomes of their area, even when they’re already printed in different venues.
- Posters are one other change that now we have applied. Posters normally visually symbolize a brand new outcome or a piece in progress. We now enable a small portion of “Challenge Posters”. With these “venture” posters, we’re providing the chance to presenters to share a piece or a collection of works which have already been printed (exterior of SC conferences, for instance), however that they consider would curiosity the SC group.
In each instances, the concept is to take away the “publication” factor, in order that we are able to truly concentrate on the scientific contribution and foster dialogue inside the SC group.
With respect to greatest practices and enhancing the standard of the scientific contributions, the modifications that now we have made embody:
- Reproducibility stories. The reproducibility of HPC outcomes is an advanced downside. For a number of years now now we have been awarding “reproducibility badges” to papers, based mostly on what a devoted committee was in a position to do. However these badges are very binary. As an example, what occurs if you happen to don’t have the complete scale to breed a outcome, however had been in a position to reproduce it on the scale you’ve entry to? Ought to this paper get a “outcome replicated” badge? To enhance this course of, now we have launched a “reproducibility report”, which can describe precisely what outcome was reproduced and the way. It can additionally present recognition to committee members. On a facet be aware, we’re extraordinarily grateful to Sascha Hunold (Reproducibility Chair) who has achieved an unbelievable job at implementing this and who most likely wasn’t anticipating this big workload.
- A much less seen procedural change is that now we have requested each factor to remind submitters of the definition of authorship (as defined by IEEE). Precisely representing authors is a part of tutorial integrity. In a time the place the general public’s belief in Science/Academia is reducing, it’s our accountability to maintain our ethics and trustworthiness to the best requirements.
Q: Any surprising challenges with this 12 months’s Technical Program?
TP: An fascinating downside that now we have needed to cope with this 12 months is the utilization of Giant Language Fashions (LLM) in paper (and evaluation) writing. IEEE and ACM have began to draft insurance policies, however it’s nonetheless unclear how authors use them. That is one thing new that’s evolving shortly.
For Tech Papers and Workshops, now we have added a area for authors to explain if and the way they’ve used LLM in writing their work (per IEEE coverage it must also be integrated into the accepted paper).
For SC24, we plan to review how these instruments are integrated into the writing course of, and it needs to be studied over time.
One other problem that now we have confronted is the rise in variety of submissions in Tech Papers (+30%!). This has apparent implications when it comes to volunteers (you can apply here for SC25) wanted to maintain the standard of evaluation and choice that SC strives for. It additionally has implications within the variety of papers that we will settle for. Fortunately, the conference middle in Atlanta is sort of giant and we must always be capable of match extra paper classes if wanted. It additionally has many difficult implications: has the group elevated and may we count on a a lot bigger attendance in November (with direct implications on room sizes)?
Q: What weren’t you in a position to change? What’s subsequent for the group?
TP: Now we have thought of having open critiques for a very long time. Open evaluation is the method the place the scientific dialogue between (nameless) professional reviewers and paper authors is made public to the readers. This tactic is rising in lots of scientific communities, like NeurIPS, one of many main venues for ML, which has been implementing this for some years. Nature has additionally began to publish some evaluation stories since 2020.
Open critiques carry additional details about a paper. They might assist to grasp what the bounds of a contribution are, which we consider are an vital a part of the scientific course of. With the present unhappy state of scientific publishing (see here for example), the standard and availability of the evaluation are a wonderful indicator of the standard of the convention and evaluation course of.
It has additionally facet advantages – for instance, it exhibits new reviewers what is anticipated of them when reviewing an SC paper, and helps new submitters to see how a piece is evaluated.
We had hoped to implement it this 12 months, however couldn’t. One thing left to do for our successors!
